Image processing device and method

ABSTRACT

An image processing device for encoding a video signal including circuitry configured to perform an arithmetic encoding process on a top block of a current block line of the video signal using a context used in the arithmetic encoding process for a previous block of a previous block line.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Ser. No. 16/153,904, filedOct. 8, 2018, which is a continuation of U.S. Ser. No. 14/874,890, filedOct. 5, 2015 (now U.S. Pat. No. 10,187,645), which is a continuation ofSer. No. 14/331,964, filed Jul. 15, 2014 (now U.S. Pat. No. 9,204,152),which is a continuation of U.S. Ser. No. 13/639,056, filed Oct. 2, 2012(now U.S. Pat. No. 8,824,813), and claims the benefit of priority from,PCT Application No. PCT/JP11/058166, filed Mar. 31, 2011, and is furtherbased upon, and claims the benefit of priority under 35 U.S.C. § 119from, Japanese Patent Application No. 2010-090960, filed Apr. 9, 2010,the entire content of each is incorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to an image processing device and method,and more particularly relates to an image processing device and methodenabling deterioration in encoding efficiency due to increased speed ofimage encoding to be suppressed.

BACKGROUND ART

Conventionally, with the AVC (Advanced Video Coding) image encodingformat, CABAC (Context-based Adaptive Binary Arithmetic Coding) andCAVLC (Context-based Adaptive Variable Length Coding) have been definedas entropy encoding. Of these, CABAC is a binary arithmetic encodingformat which performs encoding adaptively in accordance with surroundingsituations (context (Context)).

With arithmetic encoding, processing is repeated in which a numericsection is divided in accordance with occurrence probability for eachsingle symbol, thereby obtaining a code string. That is to say, there isthe need to sequentially process all symbols, and since parallelprocessing has been difficult, making the processing high-speed has beendifficult.

Accordingly, with AVC, enabling high throughput has been realized usinga technique called multi slice (multi slice). With multi slice, an imageis divided into multiple regions as shown in A in FIG. 1, and CABACencoding processing is independently performed on each. That is to say,as indicated by the dotted arrows in A in FIG. 1, arithmetic encoding isperformed for each macroblock, in the order heading from a macroblockline at the upper side within the image toward a macroblock line at thelower side within the image. Thus, CABAC encoding can be performed inparallel for each region, thereby realized high-speed processing. Thesedivided regions are called slices (slice).

The macroblocks are each encoded while referencing other adjacentmacroblocks, as shown in B in FIG. 1 and C in FIG. 1.

However, dividing regions is a factor in reducing encoding efficiency.Firstly, each slice is independently encoded, so as shown in B in FIG. 1and C in FIG. 1, mutual reference cannot be performed in the encoding ofeach region (slice). Accordingly, intra-screen prediction(intra-prediction) at the boundary portion cannot be used, and there hasbeen the concern that encoding efficiency would deteriorate.

Secondly, with CABAC, the state of surrounding macroblocks is referencedto select a symbol occurrence probability table, as shown in C in FIG.1, but since reference of adjacent macroblocks is impossible at theboundary portion, there has been the concern that selection of asuitable occurrence probability table would be difficult.

Thirdly, the symbol occurrence probability table used with CABACconverges on actual occurrence probability as encoding progresses andthe encoding efficiency increases, but with the case of multi slice(multi slice), the symbol occurrence probability table is initializedeach region (slice), so there has been the concern that the encodingefficiency at the start point of the slice (slice) would deteriorate.

There has been proposed an entropy slice (entropy slice) format as atechnique for improving encoding efficiency while performing CABAC inparallel (e.g., see NPL 1). According to this format, just the CABACprocessed portion is divided into regions called entropy slices (entropyslice) and executed in parallel, so intra-screen prediction could beused at the entropy slice (entropy slice) boundary as well. However,surrounding macroblock reference in CABAC processing has been unusableat the entropy slice (entropy slice) boundary.

As a further improvement technique, the ordered entropy slices (orderedentropy slices) format has been proposed (e.g., see NPL 2). According tothis format, reference of adjacent macroblocks has been enabled at theboundary portion for CABAC processing as well.

CITATION LIST Non Patent Literature

-   NPL 1: A. Segall, J. Zhao, “Entropy slices for parallel entropy    decoding”, VCEG input document COM16-C405, Geneva, CH, April 2008-   NPL 2: Xun Guo, Yu-Wen Huang, Shawmin Lei “Ordered Entropy Slices    for Parallel CABAC”, VCEG input document VCEG-AK25, Yokohama, Japan,    April 2009

SUMMARY OF INVENTION Technical Problem

However, in order to encode the second entropy slice (entropy slice),the processing of macroblocks of the first entropy slice (entropy slice)which is adjacent at the top thereof must have been encoded, leading tointerdependence among slices (slice), and the merit of performing inparallel was lost. That is to say, achieving high speed with parallelexecution of CABAC processing, which is the original object, has beendifficult.

The present disclosure has been made in light of this situation, and itis an object thereof to suppress deterioration in encoding efficiencydue to high speed image encoding.

Solution to Problem

One aspect of the present disclosure is an image processing device whichencodes image data, the image processing device including: an encodingunit which, with a small region line in which are arrayed in thehorizontal direction small regions obtained by dividing large regionsarrayed in the vertical direction of an image as an object, performsencoding of image data of the large region, advancing the encoding orderof small regions in the same order as with the scan order, and advancingthe encoding order of small region lines in an order different from thescan order; and a control unit which controls the encoding unit so as toencode in accordance with a state of a left small region situated at aleft periphery of a small region to be processed, and a state of a lowersmall region situated at a lower periphery of the small region to beprocessed.

The encoding unit may, with image data of a large region situatedodd-numbered from the top of the image as an object, advance theencoding order of small regions in the same order as with the scanorder, and advance the encoding order of small region lines in an orderdifferent from the scan order.

The control unit may, with image data of a large region situatedodd-numbered from the top of the image as an object, control theencoding unit so as to encode image data of a small region to beprocessed without referencing the state of a small region situated onebelow the small region to be processed, for a small region line situatedat the bottommost position of a large region to be processed.

In the event that image data of a small region line situated at thebottommost position of a large region to be processed has been stored ina storage unit, the control unit may, with image data of a large regionsituated odd-numbered from the top of the image as an object, controlthe encoding unit so as to read out and encode the image data from thestorage unit in the order of the encoding.

The encoding unit may, with image data of a large region situatedeven-numbered from the top of the image as an object, advance theencoding order of small regions in the same order as with the scan orderfor the small region lines, and advance the encoding order of the smallregion lines in the same order as with the scan order.

The control unit may, with image data of a large region situatedeven-numbered from the top of the image as an object, for a small regionline situated at the topmost position of a large region to be processed,control the encoding unit so as to encode image data of a small regionto be processed while referencing the state of a small region of thesmall region line situated at the bottommost position in a large regionsituated one above the large region.

In the event that image data of a small region line situated at thetopmost position of a large region to be processed has been stored in astorage unit, the control unit may, with image data of a large regionsituated even-numbered from the top of the image as an object, maycontrol the encoding unit so as to read out and encode the image datafrom the storage unit in the order of the encoding.

The encoding unit may advance the encoding order of the small regionlines in the opposite order from the scan order.

The scan order may be a raster scan order.

Also, an aspect of the present disclosure is an image processing methodof an image processing device which encodes image data, wherein anencoding unit, with a small region line in which are arrayed in thehorizontal direction small regions obtained by dividing large regionsarrayed in the vertical direction of an image as an object, performsencoding of image data of the large region, advancing the encoding orderof small regions in the same order as with the scan order, and advancingthe encoding order of small region lines in an order different from thescan order; and a control unit controls the encoding unit so as toencode in accordance with a state of a left small region situated at aleft periphery of a small region to be processed, and a state of a lowersmall region situated at a lower periphery of the small region to beprocessed.

Another aspect of the present disclosure is an image processing devicewhich decodes encoded data of image data which has been encoded, theimage processing device including: a decoding unit which, with a smallregion line in which are arrayed in the horizontal direction smallregions obtained by dividing large regions arrayed in the verticaldirection of an image to which the encoded data corresponds as anobject, performs decoding of encoded data of the large region, advancingthe decoding order of small regions in the same order as with the scanorder, and advancing the decoding order of small region lines in anorder different from the scan order; and a control unit which controlsthe decoding unit so as to decode in accordance with a state of a leftsmall region situated at a left periphery of a small region to beprocessed, and a state of a lower small region situated at a lowerperiphery of the small region to be processed.

The decoding unit may, with image data of a large region situatedodd-numbered from the top of the image as an object, advance thedecoding order of small regions in the same order as with the scanorder, and advance the decoding order of small region lines in an orderdifferent from the scan order.

The control unit may, with image data of a large region situatedodd-numbered from the top of the image as an object, control thedecoding unit so as to decode image data of a small region to beprocessed without referencing the state of a small region situated onebelow the small region to be processed, for a small region line situatedat the bottommost position of a large region to be processed.

In the event that image data of a small region line situated at thebottommost position of a large region to be processed has been stored ina storage unit, the control unit may, with image data of a large regionsituated odd-numbered from the top of the image as an object, controlthe decoding unit so as to read out and decode the image data from thestorage unit in the order of the decoding.

The decoding unit may, with encoded data of a large region situatedeven-numbered from the top of the image as an object, advance thedecoding order of small regions in the same order as with the scan orderfor the small region lines, and advance the decoding order of the smallregion lines in the same order as with the scan order.

The control unit may, with encoded data of a large region situatedeven-numbered from the top of the image as an object, for a small regionline situated at the topmost position of a large region to be processed,control the decoding unit so as to decode encoded data of a small regionto be processed while referencing the state of a small region of thesmall region line situated at the bottommost position in a large regionsituated one above the large region.

In the event that image data of a small region line situated at thetopmost position of a large region to be processed has been stored in astorage unit, the control unit may, with image data of a large regionsituated even-numbered from the top of the image as an object, controlthe decoding unit so as to read out and decode the image data from thestorage unit in the order of the decoding.

The decoding unit may advance the decoding order of the small regionlines in the opposite order from the scan order.

The scan order may be a raster scan order.

Also, another aspect of the present disclosure is an image processingmethod of an image processing device which decodes encoded data of imagedata which has been encoded, wherein a decoding unit, with a smallregion line in which are arrayed in the horizontal direction smallregions obtained by dividing large regions arrayed in the verticaldirection of an image to which the encoded data corresponds as anobject, performs decoding of encoded data of the large region, advancingthe decoding order of small regions in the same order as with the scanorder, and advancing the decoding order of small region lines in anorder different from the scan order; and a control unit controls thedecoding unit so as to decode in accordance with a state of a left smallregion situated at a left periphery of a small region to be processed,and a state of a lower small region situated at a lower periphery of thesmall region to be processed.

With one aspect of the present disclosure, with a small region line inwhich are arrayed in the horizontal direction small regions obtained bydividing large regions arrayed in the vertical direction of an image asan object, encoding is performed of image data of the large region,advancing the encoding order of small regions in the same order as withthe scan order, and advancing the encoding order of small region linesin an order different from the scan order, with an encoding unit beingcontrolled so as to encode in accordance with a state of a left smallregion situated at a left periphery of a small region to be processed,and a state of a lower small region situated at a lower periphery of thesmall region to be processed.

With another aspect of the present disclosure, with a small region linein which are arrayed in the horizontal direction small regions obtainedby dividing large regions arrayed in the vertical direction of an imageto which the encoded data corresponds as an object, decoding isperformed of encoded data of the large region, advancing the decodingorder of small regions in the same order as with the scan order, andadvancing the decoding order of small region lines in an order differentfrom the scan order, with a decoding unit being controlled so as todecode in accordance with a state of a left small region situated at aleft periphery of a small region to be processed, and a state of a lowersmall region situated at a lower periphery of the small region to beprocessed.

Advantageous Effects of Invention

According to the present disclosure, image data can be encoded.Particularly, deterioration in encoding efficiency due to high speedencoding of image encoding can be suppressed.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram describing an example of conventional processingprogression direction and reference direction.

FIG. 2 is a block diagram illustrating a primary configuration exampleof an image encoding device.

FIG. 3 is a block diagram illustrating a primary configuration exampleof a lossless encoding unit.

FIG. 4 is a diagram describing an example of processing progressiondirection and reference direction.

FIG. 5 is a flowchart describing an example of the flow of encodingprocessing.

FIG. 6 is a flowchart describing an example of the flow of losslessencoding processing.

FIG. 7 is a flowchart describing an example of the flow of encodingprocessing of slice 1.

FIG. 8 is a flowchart describing an example of the flow of encodingprocessing of slice 2.

FIG. 9 is a block diagram illustrating a primary configuration exampleof an image decoding device.

FIG. 10 is a block diagram illustrating a primary configuration exampleof a lossless decoding unit.

FIG. 11 is a flowchart describing an example of the flow of decodingprocessing.

FIG. 12 is a flowchart describing an example of the flow of losslessdecoding processing.

FIG. 13 is a flowchart describing an example of the flow of decodingprocessing of slice 1.

FIG. 14 is a flowchart describing an example of the flow of decodingprocessing of slice 2.

FIG. 15 is a block diagram illustrating another configuration example ofa lossless encoding unit.

FIG. 16 is a flowchart describing an example of the flow of losslessencoding processing.

FIG. 17 is a flowchart describing another example of the flow ofencoding processing of slice 1.

FIG. 18 is a flowchart describing another example of the flow ofencoding processing of slice 2.

FIG. 19 is a diagram describing an example of the way in which aprobability table is used.

FIG. 20 is a flowchart describing another example of the flow ofencoding processing of slice 1.

FIG. 21 is a flowchart describing another example of the flow ofencoding processing of slice 2.

FIG. 22 is a diagram describing an example of four-way division.

FIG. 23 is a block diagram illustrating another configuration example ofa lossless encoding unit.

FIG. 24 is a block diagram illustrating another configuration example ofa lossless decoding unit.

FIG. 25 is a diagram illustrating another example of macroblocks.

FIG. 26 is a block diagram illustrating a primary configuration exampleof a personal computer.

FIG. 27 is a block diagram illustrating a primary configuration exampleof a television receiver.

FIG. 28 is a block diagram illustrating a primary configuration exampleof a cellular phone.

FIG. 29 is a block diagram illustrating a primary configuration exampleof a hard disk recorder.

FIG. 30 is a block diagram illustrating a primary configuration exampleof a camera.

DESCRIPTION OF EMBODIMENTS

Hereinafter, embodiments of the present invention (hereinafter referredto as embodiments) will be described. Note that description will proceedin the following order.

1. First Embodiment (image encoding device)2. Second Embodiment (image decoding device)3. Third Embodiment (image encoding device/image decoding device)4. Fourth Embodiment (personal computer)5. Fifth Embodiment (television receiver)6. Sixth Embodiment (cellular telephone)7. Seventh Embodiment (hard disk recorder)8. Eighth Embodiment (camera)

1. First Embodiment [Image Encoding Device]

FIG. 2 represents the configuration of an embodiment of an imageencoding device serving as an image processing device.

An image encoding device 100 shown in FIG. 2 is an encoding device whichsubjects an image to compression encoding using, for example, the H.264and MPEG (Moving Picture Experts Group) 4 Part 10 (AVC (Advanced VideoCoding)) (hereafter, called H.264/AVC) format. Note however, that theimage encoding device 100 performs CABAC with the multi slice format asa lossless encoding format.

With the example in FIG. 2, the image encoding device 100 has an A/D(Analog/Digital) conversion unit 101, a screen rearranging buffer 102, acomputing unit 103, an orthogonal transform unit 104, a quantizationunit 105, a lossless encoding unit 106, and a storage buffer 107. Theimage encoding device 100 also has an inverse quantization unit 108, aninverse orthogonal transform unit 109, and a computing unit 110.Further, the image encoding device 100 has a deblocking filter 111, andframe memory 112. Also, the image encoding device 100 has a selectingunit 113, an intra prediction unit 114, a motion prediction/compensationunit 115, and a selecting unit 116. Further, the image encoding device100 has a rate control unit 117.

The A/D conversion unit 101 performs A/D conversion of input image data,and outputs to the screen rearranging buffer 102 and stores. The screenrearranging buffer 102 rearranges the images of frames in the storedorder for display into the order of frames for encoding according to GOP(Group of Picture) structure. The screen rearranging buffer 102 suppliesthe images of which the frame order has been rearranged to the computingunit 103, intra prediction unit 114, and motion prediction/compensationunit 115.

The computing unit 103 subtracts, from the image read out from thescreen rearranging buffer 102, the prediction image supplied from theselecting unit 116, and outputs difference information thereof to theorthogonal transform unit 104. For example, in the case of an imageregarding which intra encoding has performed, the computing unit 103adds the prediction image supplied from the intra prediction unit 114 tothe image read out from the screen rearranging buffer 102. Also, forexample, in the case of an image regarding which inter encoding has beenperformed, the computing unit 103 adds the prediction image suppliedfrom the motion prediction/compensation unit 115 to the image read outfrom the screen rearranging buffer 102.

The orthogonal transform unit 104 subjects the difference informationfrom the computing unit 103 to orthogonal transform, such as discretecosine transform, Karhunen-Loéve transform, or the like, and supplies atransform coefficient thereof to the quantization unit 105. Thequantization unit 105 quantizes the transform coefficient that theorthogonal transform unit 104 outputs. The quantization unit 105supplies the quantized transform coefficient to the lossless encodingunit 106.

The lossless encoding unit 106 subjects the quantized transformcoefficient to CABAC with the multi slice format. That is to say, thelossless encoding unit 106 divides a quantized transform coefficientimage region into the two of upper and lower entropy slices (slice 1 andslice 2), and performs CABAC on each slice parallel with each other.

At this time, the lossless encoding device 106 performs CABAC as to theentropy slice at the upper side (slice 1) in the order heading fromlower macroblocks toward upper macroblocks. Also, the lossless encodingdevice 106 performs CABAC as to the entropy slice at the lower side(slice 2) in the order heading from upper macroblocks toward lowermacroblocks.

Thus, the lossless encoding unit 106 sequentially processes themacroblocks of the two upper and lower entropy slices in directions eachgoing away from the boundary therebetween.

The lossless encoding unit 106 obtains information indicating intraprediction and so forth from the intra prediction unit 114, and obtainsinformation indicating an inter prediction mode and so forth from themotion prediction/compensation unit 115. Note that the informationindicating intra prediction will also be referred to as intra predictionmode information hereinafter. Also, the information indicatinginformation mode indicating inter prediction will also be referred to asinter prediction mode information hereinafter.

The lossless encoding unit 106 encodes the quantized transformcoefficient, and also takes filter coefficients, intra prediction modeinformation, inter prediction mode information, quantization parameters,and so forth, as part of header information in the encoded data(multiplexes). The lossless encoding unit 106 supplies the encoded dataobtained by encoding to the storage buffer 107 for storage.

The storage buffer 107 temporarily holds the encoded data supplied fromthe lossless encoding unit 106, and at a predetermined timing outputsthis to, for example, a recording device or transmission path or thelike downstream not shown in the drawing, as an encoded image encoded bythe H.264/AVC format.

Also, the quantized transform coefficient output from the quantizationunit 105 is also supplied to the inverse quantization unit 108. Theinverse quantization unit 108 performs inverse quantization of thequantized transform coefficient with a method corresponding toquantization at the quantization unit 105, and supplies the obtainedtransform coefficient to the inverse orthogonal transform unit 109.

The inverse orthogonal transform unit 109 performs inverse orthogonaltransform of the supplied transform coefficients with a methodcorresponding to the orthogonal transform processing by the orthogonaltransform unit 104. The output subjected to inverse orthogonal transformis supplied to the computing unit 110.

The computing unit 110 adds the inverse orthogonal transform resultsupplied from the inverse orthogonal transform unit 109, i.e., therestored difference information, to the prediction image supplied fromthe selecting unit 116, and obtains a locally decoded image (decodedimage). In the event that the difference information corresponds to animage regarding which intra encoding is to be performed, for example,the computing unit 110 adds the prediction image supplied from the intraprediction unit 114 to that difference information. Also, in the eventthat the difference information corresponds to an image regarding whichinter encoding is to be performed, for example, the computing unit 110adds the prediction image supplied from the motionprediction/compensation unit 115 to that difference information.

The addition results thereof are supplied to the deblocking filter 111or frame memory 112.

The deblocking filter 111 removes block noise from the decoded image byperforming deblocking filter processing as appropriate, and alsoperforms image quality improvement by performing loop filter processingas appropriate using a Wiener filter (Wiener Filter), for example. Thedeblocking filter 111 performs class classification of each of thepixels, and performs appropriate filter processing for each class. Thedeblocking filter 111 then supplies the filter processing results to theframe memory 112.

The frame memory 112 outputs the stored reference image to the intraprediction unit 114 or the motion prediction/compensation unit 115 viathe selecting unit 113 at a predetermined timing.

For example, in the case of an image regarding which intra encoding isto be performed, for example, the frame memory 112 supplies thereference image to the intra prediction unit 114 via the selecting unit113. Also, in the case of an image regarding which inter encoding is tobe performed, for example, the frame memory 112 supplies the referenceimage to the motion prediction/compensation unit 115 via the selectingunit 113.

With the image encoding device 100, the I picture, B picture, and Ppicture from the screen rearranging buffer 102 are supplied to the intraprediction unit 114 as an image to be subjected to intra prediction(also referred to as intra processing), for example. Also, the B pictureand P picture read out from the screen rearranging buffer 102 aresupplied to the motion prediction/compensation unit 115 as an image tobe subjected to inter prediction (also referred to as inter processing).

The selecting unit 113 supplies the reference image supplied from theframe memory 112 to the intra prediction unit 114 in the case of animage regarding which intra encoding is to be performed, and supplies tothe motion prediction/compensation unit 115 in the case of an imageregarding which inter encoding is to be performed.

The intra prediction unit 114 performs intra prediction to generate aprediction image using pixel values within the screen (intra screenprediction). The intra prediction unit 114 performs intra prediction bymultiple modes (intra prediction modes). The intra prediction modesinclude a mode for generating a prediction image based on a referenceimage supplied from the frame memory 112 via the selecting unit 113.

The intra prediction unit 114 generates prediction images in all intraprediction modes, evaluates the prediction images, and selects anoptimal mode. Upon selecting an optimal intra prediction mode, the intraprediction unit 114 supplies the prediction image generated in thatoptimal mode to the calculating unit 103 via the selecting unit 116.Also, as described above, the intra prediction unit 114 suppliesinformation such as intra prediction mode information indicating theintra prediction mode employed, and so forth, to the lossless encodingunit 106 as appropriate.

With regard to the image to be subjected to inter encoding, the motionprediction/compensation unit 115 uses the input image supplied from thescreen rearranging buffer 102 and decoded image serving as the referenceframe supplied from the frame memory 112 via the selecting unit 113, andcalculates a motion vector. The motion prediction/compensation unit 115performs motion compensation processing according to the calculatedmotion vector, and generates a prediction image (inter prediction imageinformation).

The motion prediction/compensation unit 115 performs inter predictionprocessing for all candidate inter prediction modes, and generatesprediction images. The motion prediction/compensation unit 115 suppliesthe generated prediction image to the computing unit 103 via theselecting unit 116.

The motion prediction/compensation unit 115 supplies inter predictionmode information indicating the inter prediction mode that has beenemployed, and the motion vector information indicating the calculatedmotion vector, to the lossless encoding unit 106.

The selecting unit 116 supplies the output of the intra prediction unit114 to the computing unit 103 in the case of an image for performingintra encoding, and supplies the output of the motionprediction/compensation unit 115 to the computing unit 103 in the caseof an image for performing inter encoding.

The rate control unit 117 controls the rate of quantization operationsof the quantization unit 105 based on the compressed image stored in thestorage buffer 107, such that overflow or underflow does not occur.

[Configuration of Lossless Encoding Unit]

FIG. 3 is a block diagram illustrating a primary configuration exampleof the lossless encoding unit 106. As shown in FIG. 3, the losslessencoding unit 106 includes a control unit 151, a storage unit 152, aslice 1 processing unit 153, and a slice 2 processing unit 154.

The control unit 151 controls input/output of data of the storage unit152. The control unit 151 has a macroblock detecting unit 161 and aslice detecting unit 162.

Coefficient data is supplied from the quantization unit 105 for eachmacroblock. For example, as indicated by the solid arrows in A in FIG.1, coefficient data is supplied one macroblock at a time in the orderheading from macroblocks at the left toward macroblocks at the right, inthe horizontal direction array of the macroblocks (hereinafter referredto as macroblock line). Also, as indicated by the dotted arrows in A inFIG. 1, supply is performed with greater priority the higher in theimage the macroblock line is.

That is to say, the coefficient data of one image (one picture or onefield) is supplied from the macroblock at the upper left edge within theimage, and the macroblock at the lower right edge is supplied last.

The macroblock detecting unit 161 detects each macroblock of coefficientdata supplied from the quantization unit 105 in such an order, andstores the head address of each macroblock line in the storage unit 152along with the coefficient data. Also, the macroblock detecting unit 161notifies the slice detecting unit 162 of the head address of themacroblock line that has been detected.

The slice detecting unit 162 detects the boundary between entropy slices(the boundary between slice 1 and slice 2) from the head address of themacroblock line, and operates the slice 1 processing unit 153 and slice2 processing unit 154 in accordance with these detection results.

For example, as shown in A in FIG. 1, in the event that an image (onepicture or one field) is to be divided into two, slice 1 and slice 2,the slice detecting unit 162 detects from the head address of themacroblock line that the coefficient data of the upper half of the imagehas been stored in the storage unit 152, and detects the boundary ofentropy slices.

Upon determining that the writing of the coefficient data of thebottommost macroblock line of slice 1 has started, the slice detectingunit 162 operates the slice 1 processing unit 153, and starts reading ofthe coefficient data of slice 1 from the storage unit 152.

Also, upon determining that the writing of the coefficient data onemacroblock line below that has started, the slice detecting unit 162operates the slice 2 processing unit 154 as well, and starts reading ofthe coefficient data of slice 2 from the storage unit 152 as well.

The storage unit 152 has a storage medium of an optional type, examplesof which include semiconductor memory such as RAM (Random AccessMemory), flash memory, a hard disk, and so forth, and stores informationsuch as coefficient data and start address of macroblock lines and soforth. Basically, the arithmetic encoding processing (CABAC) which theslice 1 processing unit 153 and slice 2 processing unit 154 performs hasa great load and processing time is long. Also, processing speedchanges. Accordingly, by buffering the coefficient data supplied by thestorage unit 152, occurrence of overflow and underflow can besuppressed.

The slice 1 processing unit 153 reads out the coefficient data of slice1 from the storage unit 152, and performs CABAC. The slice 1 processingunit 153 has a readout control unit 171, macroblock line memory 172, acontext calculating unit 173, a binarization unit 174, and an arithmeticencoding unit 175.

The readout control unit 171 references the head address of themacroblock line written to a different region from the region where thecoefficient data is stored in the storage unit 152, and reads out thecoefficient data of slice 1 from the storage unit 152 for eachmacroblock. The readout control unit 171 reads out with priority thecoefficient data from the bottommost macroblock line of slice 1. Thereadout control unit 171 stores the coefficient data that has been readout in the macroblock line memory 172, and also supplies this to thebinarization unit 174.

The macroblock line memory 172 has a recording medium of an optionaltype, examples of which include semiconductor memory such as RAM (RandomAccess Memory), flash memory, and so forth, a hard disk, or the like,and stores 1 macroblock line or more of coefficient data.

The context calculating unit 173 uses the coefficient data stored in themacroblock line memory 172 to obtain the state of peripheral macroblocksadjacent to a macroblock to be processed, calculates context (Context)indicating an occurrence probability table and symbols with a highprobability of occurrence based on the peripheral state, and suppliesthese to the arithmetic encoding unit 175.

The binarization unit 174 binarizes the coefficient data (multi-valuedata) supplied from the readout control unit 171, and supplies thisbinarized data (binary data) to the arithmetic encoding unit 175.

The arithmetic encoding unit 175 performs binary arithmetic encoding ofthe binary data supplied from the binarization unit 174, following thecontext supplied from the context calculating unit 173. The arithmeticencoding unit 175 supplies the obtained encoded data to the storagebuffer 107 so as to be stored.

The slice 2 processing unit 154 primarily reads out the coefficient dataof slice 2 from the storage unit 152, and performs CABAC. The slice 2processing unit 154 basically has the same configuration as the slice 1processing unit 153. That is to say, the slice 2 processing unit 154 hasa readout control unit 181, macroblock line memory 182, a contextcalculating unit 183, a binarization unit 184, and an arithmeticencoding unit 185.

The readout control unit 181 has the same configuration as that of thereadout control unit 171, and performs the same processing. Themacroblock line memory 182 has the same configuration as the macroblockline memory 172, and performs the same processing. The contextcalculating unit 183 has the same configuration as that of the contextcalculating unit 173, and performs the same processing. The binarizationunit 184 has the same configuration as that of the binarization unit174, and performs the same processing. The arithmetic encoding unit 185has the same configuration as that of the arithmetic encoding unit 175,and performs the same processing.

Note however, with the case of slice 2, the processing order andreference direction of the macroblocks differs from the case of slice 1.

[Encoding Procedures]

FIG. 4 is a diagram describing an example of processing progressiondirection and reference direction.

CABAC of one image (one picture or one field) worth is performed in anorder indicated by arrows shown in A in FIG. 4. That is to say, theimage is divided into multiple entropy slices (large regions) arrayed inthe vertical direction, and CABAC (encoding) is performed for each ofmacroblocks (small regions) into which each entropy slice is divided inthe row/column directions.

At slice 1, the coefficient data is first encoded for the bottommostmacroblock line (small region row), and next the macroblock line oneabove is encoded, as indicated by the dotted arrows in A in FIG. 4.Thus, the object of processing is moved to the macroblock line above onerow at a time, and finally, the topmost macroblock line is encoded.

Within each macroblock line, the leftmost macroblock is first encoded,and next the macroblock one to the right is encoded, in the same way aswith a conventional arrangement, as indicated by the solid arrows in Ain FIG. 4. Thus, the processing object moves to the macroblock to theright, one at a time, and finally, the macroblock at the right edge isencoded.

As indicated by the arrows in A in FIG. 1, the coefficient data isstored by macroblocks in the storage unit 152 in the order from the topmacroblock line of the image toward the bottom macroblock line. That isto say, upon coefficient data of the leftmost macroblock of thebottommost macroblock line of slice 1 being written to the storage unit152, CABAC of slice 1 is started.

In the case of conventional CABAC, a macroblock (mbA) one to the left ofthe macroblock (curr) to be processed, and a macroblock (mbB) onethereabove are referenced as peripheral macroblocks, as illustrated in Cin FIG. 1. However, in the case of this slice 1, a macroblock (mbA) oneto the left of the macroblock (curr) to be processed, and a macroblock(mbB′) one below are referenced as peripheral macroblocks, asillustrated in B in FIG. 4.

Note however, that the macroblock line one below the bottommostmacroblock line to be encoded first is a macroblock line of slice 2, andaccordingly cannot be referenced, as shown in C in FIG. 4. In this case,processing is performed in the same way as the case of the topmostmacroblock line in conventional CABAC, albeit in a different referencedirection.

Also, as shown in C in FIG. 4, in the event that the object ofprocessing is a macroblock at the left edge of the screen, a macroblock(mbA) one to the left thereof cannot be referenced. In this case,processing is performed in the same way as the case of conventionalCABAC.

Conversely, with slice 2, encoding is advanced in the same way as withthe conventional case, as indicated by the arrows in A in FIG. 4. Thatis to say, as indicated by the dotted arrows in A in FIG. 4, for thecoefficient data, the topmost macroblock line is first encoded, and nextthe macroblock line one below is encoded. Thus, the object of processingis moved down one macroblock line at a time, and finally, the bottommostmacroblock line is encoded.

Within each macroblock line, as indicated by the solid arrows in A inFIG. 4, the macroblock at the left edge is first encoded, and next themacroblock one to the right is encoded. Thus, the object of processingis moved to the right one macroblock at a time, and finally, themacroblock at the right edge is encoded.

In the case of this slice 2, the reference direction of the peripheralmacroblocks is the same as with the conventional case, as shown in D inFIG. 4, with the macroblock (mbA) one to the left of the macroblock(curr) to be processed and the macroblock (mbB) one thereabove arereferenced as peripheral macroblocks.

Accordingly, the CABAC of slice 2 can be started at the point that thecoefficient data of the macroblock at the far left of the topmostmacroblock line of slice 2 has been written to the storage unit 152.That is to say, the CABAC of slice 2 can be started without waiting forthe CABAC of slice 1 to end. For example, the CABAC of slice 2 can beexecuted in parallel with the CABAC of slice 1.

The CABAC of slice 1 does not reference the coefficient data of slice 2.Also, as described later, while the bottommost macroblock line of slice1 is referenced with regard to the topmost macroblock line of slice 2,this coefficient data of slice 1 exists in the storage unit 152 at thepoint that the CABAC of slice 2 is started. Accordingly, processing ofthe CABAC of slice 1 and the CABAC of slice 2 can advance independent ofeach other, without waiting up or the like for the processing of eachother.

Accordingly, the lossless encoding unit 106 can improve the throughputof the encoding processing. In other words, the lossless encoding unit106 can perform encoding at higher speeds.

Also, as described above, the macroblock line one above that isreferenced at the time of CABAC of the topmost macroblock line of slice2 is the bottommost macroblock line of slice 1, and is already stored inthe storage unit 152. Accordingly, as shown in D in FIG. 4, in the caseof CABAC of slice 2, the macroblock (mbB) one above can be referencedfor the first macroblock line, i.e., for encoding to the topmostmacroblock line.

More specifically, the readout control unit 181 of the slice 2processing unit 154 reads out the bottommost macroblock line of slice 1and holds this in the macroblock line memory 182, before reading out thecoefficient data of the topmost macroblock line of slice 2 as the objectof processing.

The context calculating unit 183 calculates the context of theperipheral macroblocks using coefficient data and the like of thebottommost macroblock line of slice 1 stored in the macroblock linememory 182, and provides this as to the arithmetic encoding processingof the coefficient data of the topmost macroblock line which isperformed by the arithmetic encoding unit 185.

The arithmetic encoding unit 185 uses the context thereof to performarithmetic encoding processing as to the coefficient data of the topmostmacroblock line of slice 2.

In this way, the peripheral macroblocks can be referenced at the entropyslice boundary as well, so the lossless encoding unit 106 can suppressdeterioration in encoding efficiency due to image division (multislicing).

[Encoding Processing]

The flow of processing executed by the above image encoding device 100will be described next. First, an example of the flow of encodingprocessing will be described with reference to the flowchart in FIG. 5.

In step S101, the A/D conversion unit 101 performs A/D conversion of aninput image. In step S102, the screen rearranging buffer 102 stores theimage supplied from the A/D conversion unit 101, and performsrearranging from the sequence for displaying the pictures to thesequence for encoding.

In step S103, the intra prediction unit 114 and motionprediction/compensation unit 115 each perform prediction processing forthe image. That is to say, in step S103, the intra prediction unit 114performs intra prediction mode intra prediction processing. The motionprediction/compensation unit 115 performs inter prediction mode motionprediction/compensation processing.

In step S104, the selecting unit 116 decides the optimal prediction modebased on the cost function values output from the intra prediction unit114 and motion prediction/compensation unit 115. That is to say, theselecting unit 116 selects one or the other of the prediction imagegenerated by the intra prediction unit 114 and the prediction imagegenerated by the motion prediction/compensation unit 115.

Also, the selection information of the prediction image is supplied tothe intra prediction unit 114 or motion prediction/compensation unit115. In the event that the prediction image of the optimal intraprediction mode has been selected, the intra prediction unit 114supplies information indicating the optimal intra prediction mode (i.e.,intra prediction mode information) to the lossless encoding unit 106.

In the event that the prediction image of the optimal inter predictionmode has been selected, the motion prediction/compensation unit 115outputs information indicating the optimal inter prediction mode, andinformation according to the optimal inter prediction mode as necessary,to the lossless encoding unit 106. Examples of information according tothe optimal inter prediction mode include motion vector information,flag information, reference frame information, and so forth.

In step S105, the computing unit 103 computes difference between animage rearranged in step S102 and the prediction image obtained by theprediction processing in step S103. The prediction image is supplied tothe computing unit 103 from the motion prediction/compensation unit 115in the event of performing inter prediction, and from the intraprediction unit 114 in the event of performing intra prediction, via theselecting unit 116 in either case.

The difference data is reduced in data amount as compared to theoriginal image data. Accordingly, the data amount can be compressed ascompared to the case of encoding the original image without change.

In step S106, the orthogonal transform unit 104 subjects the differenceinformation supplied from the computing unit 103 to orthogonaltransform. Specifically, orthogonal transform, such as discrete cosinetransform, Karhunen-Loéve transform, or the like, is performed, and atransform coefficient is output. In step S107, the quantization unit 105quantizes the transform coefficient.

In step S108, the lossless encoding unit 106 encodes the quantizedtransform coefficient output from the quantization unit 105. That is tosay, the difference image (secondary difference image in the case ofinter) is subjected to lossless encoding such as arithmetic encoding orthe like. Details of encoding processing will be described later.

Note that the lossless encoding unit 106 encodes information relating tothe prediction mode of the prediction image selected by the processingin step S104, and adds to the header information of encoded dataobtained by encoding the difference image.

That is to say, the lossless encoding unit 106 also encodes intraprediction mode information supplied from the intra prediction unit 114or information according to the optimal inter prediction mode suppliedfrom the motion prediction/compensation unit 115 and so forth, and addsthis to the header information.

In step S109, the storage buffer 107 stores encoded data output from thelossless encoding unit 106. The encoded data stored in the storagebuffer 107 is read out as suitable, and transmitted to the decoding sidevia the transmission path.

In step S110, the rate control unit 117 controls the rate of thequantization operation of the quantization unit 105, based on thecompressed image stored in the storage buffer 107, so as not to causeoverflow or underflow.

Also, the difference information quantized by the processing in stepS107 is locally decoded as follows. That is to say, in step S111, theinverse quantization unit 108 subjects the transform coefficientquantized by the quantization unit 105 to inverse quantization using aproperty corresponding to the property of the quantization unit 105. Instep S112, the inverse orthogonal transform unit 109 subjects thetransform coefficient subjected to inverse quantization by the inversequantization unit 108 to inverse orthogonal transform using a propertycorresponding to the property of the orthogonal transform unit 104.

In step S113, the computing unit 110 adds the prediction image input viathe selecting unit 116 to the locally decoded difference information,and generates a locally decoded image (the image corresponding to theinput to the computing unit 103). In step S114, the deblocking filter111 subjects the image output from the computing unit 110 to filtering.Thus, block distortion is removed. In step S115, the frame memory 112stores the image subjected to filtering. Note that an image notsubjected to filtering processing by the deblocking filter 111 is alsosupplied from the computing unit 110 to the frame memory 112 forstoring.

[Lossless Encoding Processing]

Next, an example of the flow of lossless encoding processing executed instep S108 in FIG. 5 will be described with reference to the flowchart inFIG. 6.

Upon the lossless encoding processing being started, in step S131, thestorage unit 152 stores the coefficient data of each macroblock suppliedvia the macroblock detecting unit 161. In step S132, the storage unit152 stores the head address of the macroblock line detected by themacroblock detecting unit 161 in a region different from the coefficientdata.

In step S133, the slice detecting unit 162 determines whether or not themacroblock stored in the storage unit 152 is a macroblock at an entropyslice boundary, and in the event that determination is made that this isan entropy slice boundary, the processing advances to step S134.

In step S134, at the point that the macroblock at the far left of thebottommost macroblock line of slice 1 has been written to the storageunit 152, the slice detecting unit 162 controls the slice 1 processingunit 153 so as to start encoding from the bottom of slice 1 toward thetop (latter-described slice 1 encoding processing). That is to say, theslice detecting unit 162 causes the slice 1 processing unit 153 toperform CABAC heading from the bottom to the top in slice 1 as describedwith reference to FIG. 4, as a separate task.

In step S135, at the point that the macroblock at the far left of thetopmost macroblock line of slice 2 has been written to the storage unit152, the slice detecting unit 162 controls the slice 2 processing unit154 so as to start encoding from the top of slice 2 toward the bottom(latter-described slice 2 encoding processing). That is to say, theslice detecting unit 162 causes the slice 2 processing unit 154 toperform CABAC heading from the top to the bottom in slice 2 as describedwith reference to FIG. 4, as a separate task.

Upon starting the CABAC on slice 1 and the CABAC on slice 2, the slicedetecting unit 162 advances the processing to step S136. Also, in theevent that determination is made in step S133 that this is not a sliceboundary, the slice detecting unit 162 advances the processing to stepS136.

In step S136, the macroblock detecting unit 161 determines whether ornot all macroblocks within the image (picture or field) have beenprocessed, and in the event that determination is made that anunprocessed macroblock exists the processing is returned to step S131,and the subsequent processing is repeated on the macroblock suppliednext.

Also, in the event that determination is made in step S136 that allmacroblocks have been processed, the macroblock detecting unit 161 endsthe lossless encoding processing, returns the processing to step S108 inFIG. 5, and advances the processing to step S109.

[Slice 1 Encoding Processing]

Next, and example of the flow of the slice 1 encoding processingexecuted by the slice 1 processing unit 153 will be described withreference to the flowchart in FIG. 7.

Upon the processing of step S134 in FIG. 6 being executed and the slice1 encoding processing being started, in step S151 the readout controlunit 171 of the slice 1 processing unit 153 reads out the coefficientdata of the last macroblock line in slice 1. In step S152, themacroblock line memory 172 stores the coefficient data of the macroblockline read out in step S151.

In step S153, the binarization unit 174 binarizes the coefficient dataof the macroblock line read out in step S151. In step S154, thearithmetic encoding unit 175 performs arithmetic encoding withoutreference the macroblock line below.

Thus, upon encoding all coefficient data of the bottommost macroblockline of slice 1, the slice 1 processing unit 153 advances the processingto step S155. In step S155, the readout control unit 171 of the slice 1processing unit 153 reads out the coefficient data of the macroblockline one above the macroblock line read out last time. In step S156, themacroblock line memory 172 stores the coefficient data of the macroblockline read out in step S155.

In step S157, the binarization unit 174 binarizes the coefficient dataof the macroblock line read out in step S155. In step S158, the contextcalculating unit 173 calculates the context of the macroblock to beprocessed, referencing the macroblock line one below. In step S159, thearithmetic encoding unit 175 performs arithmetic encoding using thecontext calculated in step S158.

In step S160, the readout control unit 171 determines whether or not allmacroblock lines within the image (picture or field) have beenprocessed, and in the event that an unprocessed macroblock line exists,returns the processing to step S155, and repeats the subsequentprocessing.

In the event that the processing of step S155 through step S160 isrepeatedly executed, and determination is made in step S160 that allmacroblock lines in the image have been processed, the readout controlunit 171 ends the slice 1 encoding processing.

[Slice 2 Encoding Processing]

Next, an example of the flow of slice 2 encoding processing executed bythe slice 2 processing unit 154 will be described with reference to theflowchart in FIG. 8.

Upon the processing of step S135 in FIG. 6 being executed and the slice2 encoding processing starting, in step S181 the readout control unit181 of the slice 2 processing unit 154 reads out the coefficient data ofthe last macroblock line in slice 1 the coefficient data of the firstmacroblock line in slice 2. In step S182, the macroblock line memory 182stores the coefficient data of the macroblock lines read out in stepS181.

In step S183, the binarization unit 184 binarizes the coefficient dataof the first macroblock line of slice 2 read out in step S181. In stepS184, the context calculating unit 183 calculates the context of themacroblock to be processed by referencing the last macroblock line ofslice 1, that was stored in step S182. In step S185, the arithmeticencoding unit 185 performs arithmetic encoding using the contextcalculated in step S184.

Thus, upon encoding all coefficient data of the topmost macroblock lineof slice 2, the slice 2 processing unit 154 advances the processing tostep S186. In step S186, the readout control unit 181 of the slice 2processing unit 154 reads out the coefficient data of the macroblockline one below the macroblock line read out last time. In step S187, themacroblock line memory 182 stores the coefficient data of the macroblockline read out in step S186.

In step S188, the binarization unit 184 binarizes the coefficient dataof the macroblock line read out in step S186. In step S189, the contextcalculating unit 183 calculates the context of the macroblock to beprocessed, referencing the macroblock line one above. In step S190, thearithmetic encoding unit 185 performs arithmetic encoding using thecontext calculated in step S189.

In step S191, the readout control unit 181 determines whether or not allmacroblock lines within the image (picture or field) have beenprocessed, and in the event that an unprocessed macroblock line exists,returns the processing to step S186, and repeats the subsequentprocessing.

In the event that the processing of step S186 through step S191 isrepeatedly executed, and determination is made in step S191 that allmacroblock lines in the image have been processed, the readout controlunit 181 ends the slice 2 encoding processing.

Thus, the lossless encoding unit 106 can realize high-speed imageencoding while suppressing deterioration in encoding efficiency.

2. Second Embodiment [Image Decoding Device]

The encoded data encoded by the image encoding device 100 described inthe first embodiment is transmitted to an image decoding devicecorresponding to the image encoding device 100 via a predeterminedtransmission path, and is decoded.

The image decoding device will be described below. FIG. 9 is a blockdiagram illustrating a primary configuration example of the imagedecoding device.

As shown in FIG. 9, an image decoding device 200 is configured of astoring buffer 201, a lossless decoding unit 202, an inversequantization unit 203, an inverse orthogonal transform unit 204, acomputing unit 205, a deblocking filter 206, a screen rearranging buffer207 and a D/A conversion unit 208. The image decoding device 200 alsohas frame memory 209, a selecting unit 210, an intra prediction unit211, a motion prediction/compensation unit 212, and a selecting unit213.

The storing buffer 201 stores encoded data transmitted thereto. Thisencoded data has been encoded by the image encoding device 100. Thelossless decoding unit 202 decodes encoded data read out from thestoring buffer 201 at a predetermined timing using a formatcorresponding to the encoding format of the lossless encoding unit 106in FIG. 2.

The inverse quantization unit 203 subjects the obtained coefficient datadecoded by the lossless decoding unit 202 to inverse quantization usinga format corresponding to the quantization format of the quantizationunit 105 in FIG. 2. The inverse quantization unit 203 supplies thecoefficient data subjected to inverse quantization to the inverseorthogonal transform unit 204. The inverse orthogonal transform unit 204subjects the coefficient data to inverse orthogonal transform using aformat corresponding to the orthogonal transform format of theorthogonal transform unit 104 in FIG. 2, and obtains decoded residualdata corresponding to the residual data before orthogonal transform atthe image encoding device 100.

The decoded residual data obtained by being subjected to inverseorthogonal transform is supplied to the computing unit 205. Also, thecomputing unit 205 is supplied with a prediction image from the intraprediction unit 211 or motion prediction/compensation unit 212, via theselecting unit 213.

The computing unit 205 adds the decoded residual data and the predictionimage, and obtains decoded image data corresponding to the image databefore subtraction of the prediction image by the computing unit 103 ofthe image encoding device 100. The computing unit 205 supplies thedecoded image data to the deblocking filter 206.

The deblocking filter 206 removes the block noise of the decoded image,and subsequently supplies this to the frame memory 209 is as to bestored and also supplies this to the screen rearranging buffer 207.

The screen rearranging buffer 207 performs rearranging of images. Thatis to say, the order of frames rearranged for encoding by the screenrearranging buffer 102 in FIG. 2 is rearranged to the original displayorder. The D/A conversion unit 208 performs D/A conversion of the imagesupplied from the screen rearranging buffer 207, outputs to an unshowndisplay, and displays.

The selecting unit 210 reads out the image for inter processing and theimage to be referenced from the frame memory, and supplies to the motionprediction/compensation unit 212. Also, the selecting unit 210 read outthe image to be used for intra prediction from the frame memory 209, andsupplies this to the intra prediction unit 211.

The intra prediction unit 211 is supplied with information indicatingintra prediction mode obtained by decoding the header information and soforth, from the lossless decoding unit 202, as appropriate. The intraprediction unit 211 generates a prediction image based on thisinformation, and supplies the generated prediction image to theselecting unit 213.

The motion prediction/compensation unit 212 obtains information obtainedby decoding the header information (prediction mode information, motionvector information, reference frame information) from the losslessdecoding unit 202. In the event that information indicating interprediction mode has been supplied, based on the inter motion vectorinformation from the lossless decoding unit 202, the motionprediction/compensation unit 212 generates a prediction image, andsupplies the generated prediction image to the selecting unit 213.

The selecting unit 213 selects a prediction image generated by themotion prediction/compensation unit 212 or the intra prediction unit211, and supplies this to the computing unit 205.

[Lossless Decoding Unit]

FIG. 10 is a block diagram illustrating a primary configuration exampleof the lossless decoding unit 202 shown in FIG. 9.

As shown in FIG. 10, the lossless decoding unit 202 has a demultiplexer251, a slice 1 processing unit 252, a slice 2 storage unit 253, astorage unit 254, and a readout control unit 255.

The demultiplexer 251 identifies the entropy slice to which the encodeddata supplied from the storage buffer 201 belongs (whether slice 1 orslice 2), and controls the supply destination of the encoded datadepending on the entropy slice.

The bit stream generated by the image encoding device 100 shown in FIG.2 is multiplexed with each entropy slice as en independent stream. Themultiplexed streams are supplied from the storage buffer 201.

For example, in the event of encoded data belonging to slice 1, thedemultiplexer 251 supplies to the slice 1 processing unit 252. Also, forexample, in the event of encoded data belonging to slice 2, thedemultiplexer 251 supplies to the slice 2 processing unit 253.

The slice 1 processing unit 252 performs arithmetic decoding of theencoded data belonging to slice 1. With the image encoding device 100,each macroblock belonging to slice 1 has been encoded from left to rightand bottom to top of the image. Also, the peripheral blocks to bereferenced are a macroblock one to the left and a macroblock one belowin the macroblock to be processed. Decoding processing is also performedin the same processing order and reference direction as this.

The slice 2 processing unit 253 performs arithmetic decoding of theencoded data belonging to slice 2. With the image encoding device 100,each macroblock belonging to slice 2 has been encoded from left to rightand top to bottom of the image. Also, the peripheral blocks to bereferenced are a macroblock one to the left and a macroblock one abovein the macroblock to be processed. Decoding processing is also performedin the same processing order and reference direction as this.

The storage unit 254 obtains the decoded coefficient data generated atthe slice 1 processing unit 252 (multi-value data) and the decodedcoefficient data generated at the slice 2 processing unit 253(multi-value data), and stores these. The readout control unit 255 readsout the decoded coefficient data stored in the storage unit 254 at apredetermined timing and in a predetermined order, and supplies these tothe inverse quantizing unit 203.

Note that the lossless decoding unit 202 actually extracts metadata suchas encoding parameters and prediction mode information and so forth fromthe supplied bit streams, and supplies these to the intra predictionunit 211 and motion prediction/compensation unit 212.

The slice 1 processing unit 252 has an arithmetic decoding unit 261, amulti-valuing unit 262, macroblock line memory 263, and a contextcalculating unit 264.

The arithmetic decoding unit 261 performs arithmetic decoding of encodeddata using context calculated by the context calculating unit 264, andgenerates binary data. The multi-valuing unit 262 performs multi-valuingof the binary data output from the arithmetic decoding unit 261. Themulti-valuing unit 262 supplies the generated multi-value data to thestorage unit 254 as decoded coefficient data, and also supplies this tothe macroblock line memory 263.

The macroblock line memory 263 stores the decoded coefficient datasupplied from the multi-valuing unit 262. The macroblock line memory 263can store one macroblock line worth or more of decoded coefficient data.

The context calculating unit 264 uses the decoded coefficient datastored in the macroblock line memory 263 to calculate the context of theperipheral macroblocks adjacent to the macroblock to be subjected todecoding processing by the arithmetic decoding unit 261, and suppliesthis to the arithmetic decoding unit 261.

For example, the context calculating unit 264 generates context of themacroblock one to the left of the macroblock to be processed, andcontext of the macroblock one below the macroblock to be processed.

The slice 2 processing unit 253 has an arithmetic decoding unit 271, amulti-valuing unit 272, macroblock line memory 273, and a contextcalculating unit 274.

The arithmetic decoding unit 271 performs arithmetic decoding of encodeddata using context calculated by the context calculating unit 274, andgenerates binary data. The multi-valuing unit 272 performs multi-valuingof the binary data output from the arithmetic decoding unit 271. Themulti-valuing unit 272 supplies the generated multi-value data to thestorage unit 254 as decoded coefficient data, and also supplies this tothe macroblock line memory 273.

The macroblock line memory 273 stores the decoded coefficient datasupplied from the multi-valuing unit 272. The macroblock line memory 273can store one macroblock line worth or more of decoded coefficient data.

The context calculating unit 274 uses the decoded coefficient datastored in the macroblock line memory 273 to calculate the context of theperipheral macroblocks adjacent to the macroblock to be subjected todecoding processing by the arithmetic decoding unit 271, and suppliesthis to the arithmetic decoding unit 271.

For example, the context calculating unit 274 generates context of themacroblock one to the left of the macroblock to be processed, andcontext of the macroblock one above the macroblock to be processed.

Note that in the event that the object of decoding processing is amacroblock line at the topmost position of slice 2, the contextcalculating unit 274 generates the context of the macroblock one abovethe macroblock to be processed, using the decoded coefficient data ofthe bottommost macroblock line of slice 1 stored in the macroblock linememory 263 of the slice 1 processing unit 252.

By performing such decoding processing as described above, the losslessdecoding unit 202 advances processing in an order the same as with thecase of the lossless encoding unit 106, and performs peripheralreference in the same direction, so the encoded data can be correctlydecoded.

[Decoding Processing]

Next, the flow of each processing executed by the above-described imagedecoding device 200 will be described. First, an example of the flow ofdecoding processing will be described with reference to the flowchart inFIG. 11.

Upon the decoding processing being started, in step S201, the storingbuffer 201 stores the transmitted encoded data. In step S202, thelossless decoding unit 202 decodes the encoded data supplied from thestoring buffer 201. Specifically, the I picture, P picture, and Bpicture encoded by the lossless encoding unit 106 in FIG. 2 are decoded.

At this time, the motion vector information, reference frameinformation, prediction mode information (intra prediction mode or interprediction mode), and flag information and so forth, are also decoded.

Specifically, in the event that the prediction mode information is intraprediction mode information, the prediction mode information is suppliedto the intra prediction unit 211. In the event that the prediction modeinformation is inter prediction mode information, prediction modeinformation and corresponding motion vector information are supplied tothe motion prediction/compensation unit 212.

In step S203, the inverse quantization unit 203 inversely quantizes thetransform coefficient decoded by the lossless decoding unit 202 using aproperty corresponding to the property of the quantization unit 105 inFIG. 2. In step S204, the inverse orthogonal transform unit 204 subjectsthe transform coefficient inversely quantized by the inversequantization unit 203 to inverse orthogonal transform using a propertycorresponding to the property of the orthogonal transform unit 104 inFIG. 2. This means that difference information corresponding to theinput of the orthogonal transform unit 104 in FIG. 2 (the output of thecomputing unit 103) has been decoded.

In step S205, the intra prediction unit 211 or motionprediction/compensation unit 212 performs the respective imageprediction processing in accordance with the prediction mode informationsupplied from the lossless decoding unit 202.

That is to say, in the event that intra prediction mode information issupplied from the lossless decoding unit 202, the intra prediction unit211 performs intra prediction mode intra prediction processing. Also, inthe event that inter prediction mode information is supplied from thelossless decoding unit 202, the motion prediction/compensation unit 212performs inter prediction mode motion prediction processing.

In step S206, the selecting unit 213 selects a prediction image. That isto say, the selecting unit 213 is supplied with a prediction imagegenerated by the intra prediction unit 211, or, a prediction imagegenerated by the motion prediction/compensation unit 212. The selectingunit 213 selects one of these. The selected prediction image is suppliedto the computing unit 205.

In step S207, the computing unit 205 adds the prediction image selectedin the processing in step S206 to the difference information obtained bythe processing in step S204. Thus, the original image data is decoded.

In step S208, the deblocking filter 206 subjects the decoded image datasupplied from the computing unit 205 to filtering. Thus, blockdistortion is removed.

In step S209, the frame memory 209 stores the decoded image datasubjected to filtering.

In step S210, the screen rearranging buffer 207 performs rearrangingframes of the decoded image data. Specifically, the sequence of framesof the decoded image data rearranged for encoding by the screenrearranging buffer 102 (FIG. 2) of the image encoding device 100 isrearranged in the original display sequence.

In step S211, the D/A conversion unit 208 performs D/A conversion of thedecoded image data from the screen rearranging buffer 207 regardingwhich the frames have been rearranged. This decoded image data is outputto an unshown display, and the image is displayed.

[Lossless Decoding Processing]

Next, an example of the flow of lossless decoding processing executed instep S202 in FIG. 11 will be described with reference to the flowchartin FIG. 12.

Upon the lossless decoding processing being started, in step S231 thedemultiplexer 251 divides the coefficient data into individual slices.In step S232, the demultiplexer 251 supplies the encoded data of slice 1to the slice 1 processing unit 252, and starts decoding heading from thebottom toward the top of slice 1 (slice 1 decoding processing) as aseparate task.

In step S233, the demultiplexer 251 determines whether or not decodingof the last (bottommost) macroblock line of slice 1 has ended with theslice 1 decoding processing started in step S232, and stands by untildetermination is made that this has ended. In the event thatdetermination is made that decoding of the last (bottommost) macroblockline of slice 1 has ended, the demultiplexer 251 advances the processingto step S234.

In step S234, the demultiplexer 251 supplies the encoded data from slice2 to the slice 2 processing unit 253 and starts decoding heading fromthe top toward the bottom of slice 2 (slice 2 decoding processing) as aseparate task.

In step S235, the demultiplexer 251 determines whether or not decodingof all macroblocks of slice 2 has ended with the slice 2 decodingprocessing started in step S234, and stands by until determination ismade that this has ended. In the event that determination is made thatdecoding of slice 2 has ended, the demultiplexer 251 ends the losslessdecoding processing, returns the processing to step S202 in FIG. 11, andadvances the processing to step S203.

[Slice 1 Decoding Processing]

Next, an example of the flow of slice 1 decoding processing started bythe processing in step S232 in FIG. 12 will be described with referenceto the flowchart in FIG. 13.

Upon the slice 1 decoding processing being started, in step S251 thearithmetic decoding unit 261 performs arithmetic decoding of eachmacroblock in the last (bottommost) macroblock line of slice 1 withoutreferencing the macroblocks of the macroblock line below.

In step S251, the multi-valuing unit 262 multi-values the binarycoefficient data obtained by decoding in step S252, and converts intomulti-value coefficient data.

In step S253, the macroblock line memory 263 stores the coefficient data(multi-value data) of the macroblock line to be processed, that isgenerated as described above. Upon storing the coefficient data of thebottommost macroblock line of slice 1, the macroblock line memory 263advances the processing to step S254.

Upon the processing advancing to step S254, the macroblock line to besubjected to decoding processing moves one up. In step S254, the contextcalculating unit 264 references not only the macroblock line to besubjected to decoding processing, but also the macroblock line one belowthe macroblock line to be subjected to decoding processing, andcalculates context.

In step S255, the arithmetic decoding unit 261 performs arithmeticdecoding using the context generated in step S254. In step S256, themulti-valuing unit 262 multi-values the binary coefficient data obtainedby decoding in step S255, and generates multi-value data.

In step S257, the macroblock line memory 263 stores the coefficient data(multi-value data) of the macroblock line to be processed that isgenerated as described above. Upon storing the coefficient data of themacroblock line to be processed, the macroblock line memory 263 advancesthe processing to step S258.

In step S258, the slice 1 processing unit 252 determines whether or notall macroblock lines in slice 1 have been processed. In the event thatdetermination is made that there exists an unprocessed macroblock line,the slice 1 processing unit 252 returns the processing to step S254,takes the macroblock line one above the macroblock line taken as theobject of processing the previous time, and repeats the processing.

The processing of step S254 through step S258 is repeatedly performed,and in the event that determination is made in step S258 that allmacroblock lines have been processed, the slice 1 decoding processing isended.

[Slice 2 Decoding Processing]

Next, an example of the flow of slice 2 decoding processing started bythe processing in step S234 in FIG. 12 will be described with referenceto the flowchart in FIG. 14.

Upon the slice 2 decoding processing being started, in step S271 thecontext calculating unit 274 calculates context for each macroblock inthe first (topmost) macroblock line in slice 2 to be processed,referencing not only that macroblock line to be processed but also thelast (bottommost) macroblock line of slice 1 stored in the macroblockline memory 263 as well.

In step S272, the arithmetic decoding unit 271 performs arithmeticdecoding using the context calculated in step S271. In step S273, themulti-valuing unit 272 multi-values the binary coefficient data obtainedby decoding in step S272, and converts into multi-value coefficientdata.

In step S274, the macroblock line memory 273 stores the coefficient data(multi-value data) of the macroblock line to be processed, that isgenerated as described above. Upon storing the coefficient data of thetopmost macroblock line of slice 2, the macroblock line memory 273advances the processing to step S275.

Upon the processing advancing to step S275, the macroblock line to besubjected to decoding processing moves one down. In step S275, thecontext calculating unit 274 references not only the macroblock line tobe subjected to decoding processing, but also the macroblock line oneabove the macroblock line to be subjected to decoding processing, andcalculates context.

In step S276, the arithmetic decoding unit 271 performs arithmeticdecoding using the context generated in step S275. In step S277, themulti-valuing unit 272 multi-values the binary coefficient data obtainedby decoding in step S276, and generates multi-value data.

In step S278, the macroblock line memory 273 stores the coefficient data(multi-value data) of the macroblock line to be processed that isgenerated as described above. Upon storing the coefficient data of themacroblock line to be processed, the macroblock line memory 273 advancesthe processing to step S279.

In step S279, the slice 2 processing unit 253 determines whether or notall macroblock lines in slice 2 have been processed. In the event thatdetermination is made that there exists an unprocessed macroblock line,the slice 2 processing unit 253 returns the processing to step S275,takes the macroblock line one below the macroblock line taken as theobject of processing the previous time, and repeats the processing.

The processing of step S275 through step S279 is repeatedly performed,and in the event that determination is made in step S279 that allmacroblock lines have been processed, the slice 2 decoding processing isended.

Thus, the image decoding device 200 can correctly decoding encoded dataobtained by being encoded by the image encoding device 100. Accordingly,the lossless decoding unit 202 realize high speed image encoding whilesuppressing deterioration in encoding efficiency.

3. Third Embodiment [Configuration of Lossless Encoding Unit]

Now, while generating of context has been described above as beingperformed at the slice 1 processing unit 153 and the slice 2 processingunit 154, context may be generated before stored in the storage unit152, unrestricted to this.

FIG. 15 is a block diagram illustrating another configuration example ofthe lossless encoding unit 106 in FIG. 1.

As shown in FIG. 15, in this case, the lossless encoding unit 106 has,in addition to the configuration of the case in FIG. 3, a contextprocessing unit 301. Also, the lossless encoding unit 106 has a slice 1processing unit 303 instead of the slice 1 processing unit 153 in thecase in FIG. 3, and further has a slice 2 processing unit 304 instead ofthe slice 2 processing unit 154.

The context processing unit 301 calculates context form the coefficientdata supplied from the quantization unit 105. The slice 1 processingunit 303 basically has the same configuration as with the slice 1processing unit 153, but since the context processing unit 301 existsexternally from the slice 1 processing unit 303, the macroblock linememory 172 and context calculating unit 173 are omitted. The slice 2processing unit 304 basically has the same configuration as with theslice 2 processing unit 154, but since the context processing unit 301exists externally from the slice 2 processing unit 304, the macroblockline memory 182 and context calculating unit 183 are omitted.

The context processing unit 301 has macroblock line memory 311 and acontext calculating unit 312. The macroblock line memory 311 isbasically the same as with the macroblock line memory 172 and macroblockline memory 182, and stores coefficient data supplied from thequantization unit 105. Note that the reference direction of slice 1 isopposite to the conventional, and the macroblock line one below themacroblock to be processed is referenced, so the macroblock line memory311 needs to be arranged to store at least all coefficient data of onemacroblock line.

The context calculating unit 312 is basically the same as with thecontext calculating unit 173 and context calculating unit 183, andreferences the macroblock line stored in the macroblock line memory 311to calculate context.

In the case of calculating context with slice 2, the context calculatingunit 312 references the macroblock line one above the object ofprocessing, but in the case of calculating context in slice 1,references the macroblock line one blow the object of processing.

That is to say, in the case of calculating context in slice 2, thecontext calculating unit 312 calculates context (context index) byreferencing a macroblock (mb-A) one to the left of the macroblock to beprocessed (curr) and a macroblock one above (mb-B), in the same way aswith the conventional.

Conversely, in the case of calculating context in slice 1, the contextcalculating unit 312 calculates context (context index) by referencing amacroblock (mb-A) one to the left of the macroblock to be processed(curr) and a macroblock one below (mb-B′). That is to say, at the pointthat the macroblock one below (mb-B′) is supplied, the contextcalculating unit 312 reads out the macroblock to be processed (curr) andthe macroblock (mb-A) one to the left which are stored in the macroblockline memory 311, and calculates context (context index).

Note that at the time of generating context with the macroblock linesecond from the bottom in slice 1 as the object of processing (curr),the context calculating unit 312 generates context of the bottommostmacroblock line of slice 1 as well.

The context calculating unit 312 stores the calculated context in aregion of the storage unit 152 separate from the coefficient data andhead address of the macroblock addresses, in a manner tagged so as toindicate the correlation as to the coefficient data and head address ofthe macroblock addresses.

The readout control unit 171 of the slice 1 processing unit 303 readsout, along with the coefficient data and head address of the macroblockaddresses, necessary context as well, from the storage unit 152. Thereadout control unit 171 supplies the context that has been read out tothe arithmetic encoding unit 175.

The readout control unit 181 of the slice 2 processing unit 304 readsout, along with the coefficient data and head address of the macroblockaddresses, necessary context as well, from the storage unit 152. Thereadout control unit 181 supplies the context that has been read out tothe arithmetic encoding unit 185.

[Lossless Encoding Processing]

An example of the flow of lossless encoding processing in this case willbe described with reference to the flowchart in FIG. 16. This flowchartcorresponds to the flowchart in FIG. 6.

In this case, each processing of step S301 and step S302 are executed inthe same way of each processing of step S131 and step S132 in FIG. 6.

In step S303, the context processing unit 301 determines whether or notthe macroblock line to be processed is slice 1, and in the event thatdetermination is made that this is slice 1, advances the processing tostep S304.

In step S304, the context processing unit 301 calculates contextreferencing the macroblock line one blow. Note that context of thetopmost macroblock line and bottommost macroblock line of slice 1 isgenerated as described above. Upon having generated context, the contextprocessing unit 301 advances the processing to step S306.

Also, in the event that determination is made in step S303 that themacroblock line to be processed is slice 2, the context processing unit301 advances the processing to step S305.

In step S305, the context processing unit 301 calculates contextreferencing the macroblock line one above. Note that context of thetopmost macroblock line in slice 2 is generated referencing thebottommost macroblock line of slice 1. Upon having generated context,the context processing unit 301 advances the processing to step S306.

In step S306, the storage unit 152 stores the context calculated in stepS304 or step S305.

Each processing of step S307 through step S310 is performed in the sameway as each processing of step S133 through step S136 in FIG. 6.

[Slice 1 Encoding Processing]

Next, an example of the flow of slice 1 encoding processing started fromthe processing in step S308 in FIG. 16 will be described with referenceto the flowchart in FIG. 17. Note that this flowchart corresponds to theflowchart in FIG. 7.

This case is also basically performed in the same way as with FIG. 7,but the context has already been generated, so each processing of stepS152, step S156, and step S158 in FIG. 7 is omitted, and each processingof step S331 corresponding to step S151, step S332 corresponding to stepS153, step S333 corresponding to step S154, step S334 corresponding tostep S155, step S335 corresponding to step S157, step S336 correspondingto step S159, and step S337 corresponding to step S160, is executed.

Note however, that in step S334, the readout control unit 171 reads out,along with the macroblock line one above in slice 1, the correspondingcontext as well, from the storage unit 152. In step S336, the arithmeticencoding unit 175 uses the context read out in this step S334 to performarithmetic encoding.

[Slice 2 Encoding Processing]

Next, an example of the flow of slice 2 encoding processing started fromthe processing in step S309 in FIG. 16 will be described with referenceto the flowchart in FIG. 18. Note that this flowchart corresponds to theflowchart in FIG. 8.

This case is also basically performed in the same way as with FIG. 8,but the context has already been generated, so each processing of stepS182, step S184, step S186, and step S189 in FIG. 8 is omitted, and eachprocessing of step S351 corresponding to step S181, step S352corresponding to step S183, step S353 corresponding to step S185, stepS354 corresponding to step S186, step S355 corresponding to step S188,step S356 corresponding to step S190, and step S357 corresponding tostep S191, is executed.

Note however, that in step S351, the readout control unit 181 reads out,along with the first (topmost) macroblock line in slice 2, thecorresponding context as well, from the storage unit 152. That is tosay, the context of the last (bottommost) macroblock line of slice 1 isalso read out. In step S353, the arithmetic encoding unit 185 uses thecontext read out in this step S351 to perform arithmetic encoding.

In the same way, in step S354, the readout control unit 181 reads out,along with the macroblock line one below in slice 2, the correspondingcontext as well, from the storage unit 152. In step S356, the arithmeticencoding unit 185 uses the context read out in this step S354 to performarithmetic encoding.

As described above, by performing calculation of context before storingthe coefficient data in the storage unit 152, the load on the arithmeticencoding processing after readout (slice 1 processing unit 303 and slice2 processing unit 304) can be alleviated. Generally, the load ofarithmetic encoding processing is greater as compared to otherprocessing before and after this, so as described above, by calculatingcontext beforehand, before storing the coefficient data in the storageunit 152, the load can be dispersed, and reduction in costs and evenhigher speed of encoding processing and so forth can be realized.

[Duplication of Probability Table]

Further, the initial state of arithmetic encoding can be improved.

With the AVC standard, the probability table for CABAC is initializedusing a parameter called cabac_init_idc, and parameters called slicetype and SliceQPy.

In contrast with this, as shown in FIG. 19, at the point of theprocessing of the bottommost macroblock line of slice 1 ending, thecontents of the probability table which the slice 1 processing unit 153has are copied to the probability table which the slice 2 processingunit 154 has. The slice 2 processing unit 154 starts CABAC with thevalues which the slice 1 processing unit 153 had been using as theinitial value.

Thus, the lossless encoding unit 106 can start encoding with probabilityvalues more suitable for the image than the initial values according tothe standard, and encoding efficiency can be improved.

[Slice 1 Encoding Processing]

An example of the flow of the slice 1 encoding processing in this casewill be described with reference to the flowchart in FIG. 20. Theflowchart shown in FIG. 20 corresponds to the flowchart in FIG. 7.

In step S401, the arithmetic coding unit 175 initializes the probabilitytable. Each processing of step S402 through step S405 is performed inthe same way as with each processing of step S151 through step S154 inFIG. 7.

In step S406, the arithmetic coding unit 175 copies the probabilitytable of the last (rightmost) macroblock of the bottommost macroblockline, and supplies this to the arithmetic coding unit 185.

Each processing of step S407 through step S412 is performed in the sameway as with each processing of step S155 through step S160 in FIG. 7.

[Slice 2 Encoding Processing]

An example of the flow of the slice 2 encoding processing in this casewill be described with reference to the flowchart in FIG. 21. Theflowchart shown in FIG. 21 corresponds to the flowchart in FIG. 8.

Each processing of step S431 and step S432 is performed in the same wayas with each processing of step S181 and step S182 in FIG. 8.

In step S433, the arithmetic coding unit 185 obtains a copy of theprobability table of slice 1. Each processing of step S434 and step S435is performed in the same way as with each processing of step S183 andstep S184 in FIG. 8.

In step S436, the arithmetic coding unit 185 performs arithmeticencoding using context, with the copy of the probability table of slice1 as the initial value.

Each processing of step S437 through step S442 is performed in the sameway as with each processing of step S186 through step S191 in FIG. 8.

Note that with the image decoding device 200 as well, a copy of theprobability table is used as an initial value for decoding, in the sameway. Accordingly, the image encoding device 100 provides the imagedecoding device 200 with information indicating whether the probabilitytable has been initialized or copied, by adding to the encoded data orthe like, for example. Upon obtaining information to the effect that acopy of the probability table is to be used as the initial value, thelossless decoding unit 202 of the image decoding device 200 copies thecontents of the CABAC probability table at the point that the arithmeticdecoding unit 261 of the slice 1 processing unit 252 has endedprocessing of the bottommost macroblock line of slice 1, and thearithmetic decoding unit 271 of the slice 2 processing unit 253 startsthe arithmetic decoding processing using that copy. Due to such anarrangement, the lossless decoding unit 202 can use the same probabilitytable as with the above-described case of encoding, and the encoded datacan be correctly decoded.

[Multi-Division]

While description has been made above of dividing the image into two,the number of divisions may be other than two. For example, the imagemay be divided into four entropy slices of slice 1 through slice 4, asshown in FIG. 22.

In the case of dividing an image into four as with the example in FIG.22 here, encoding of slice 1 and slice 2 is performed in the same way aswith the case of division into two, described above. Also, slice 3 isencoded in the same way as with slice 1, and slice 4 in the same way aswith slice 2.

That is to say, encoding processing of slice 1 and slice 3 is advancedin the order from the bottom macroblock line toward the top macroblockline. Also, with the encoding processing of slice 1 and slice 3, themacroblock one to the left and the macroblock one below the macroblockwhich is the object of processing are referenced as peripheralmacroblocks.

On the other hand, encoding processing of slice 2 and slice 4 isadvanced in the order from the top macroblock line toward the bottommacroblock line, in the same way as with the conventional. Also, withthe encoding processing of slice 2 and slice 4, the macroblock one tothe left and the macroblock one above the macroblock which is the objectof processing are referenced as peripheral macroblocks.

In the same way as with encoding processing of slice 2 being startedwhen coefficient data of the bottommost macroblock line of slice 1 beingstored in the storage unit 152, encoding processing of slice 4 isstarted when coefficient data of the bottommost macroblock line of slice3 is stored in the storage unit 152.

Further, in the same way as the bottommost macroblock line of slice 1being referenced in the encoding processing of the topmost macroblockline of slice 2, in the encoding processing of the topmost macroblockline of slice 4, the bottommost macroblock line of slice 3 is referencedas the macroblock one above the macroblock to be processed.

[Configuration of Lossless Encoding Unit]

In the case of dividing the image into four in this way, the losslessencoding unit 106 has a slice 3 processing unit 503 which is the same asthe slice 1 processing unit 153 in FIG. 3, and a slice 4 processing unit504 which is the same as with the slice 2 processing unit 154.

Also, instead of the control unit 151 which divides the image into twoslices, the lossless encoding unit 106 has a control unit 501 whichdivides the image into four slices.

The control unit 501 has the macroblock detecting unit 161 and a slicedetecting unit 512. The slice detecting unit 512 controls the slice 1processing unit 153, slice 2 processing unit 154, slice 3 processingunit 503, and slice 4 processing unit 504, so as to divide the imageinto four slices, and causes each to perform encoding processing inparallel.

Other than the image being divided into four, each processing isexecuted in the same way as with the case of being divided into two.That is to say, slice 3 is encoded in the same way as with slice 1, andslice 4 is encoded in the same way as with slice 2.

By increasing the number of divisions in this way, the number ofparallels increases, so throughput improves, and the lossless encodingunit 106 can perform lossless encoding processing at higher speed. Atthis time, encoding processing is performed in the same way as with thecase of division into two, so the lossless encoding unit 106 can realizehigh speed image encoding while suppressing deterioration in encodingefficiency.

That is to say, the lossless encoding unit 106 of the image encodingdevice 100 divides the image to be encoded into multiple regions whichare arrayed vertically. As for the even-numbered regions from the top,the lossless encoding unit 106 then performs encoding in the order fromthe top macroblock line toward the bottom as with the conventional, andreferences the macroblock one to the left and the macroblock one abovethe macroblock to be processed, as peripheral macroblocks.

Conversely, with odd-numbered regions from the top, the losslessencoding unit 106 performs encoding in the order from the bottommacroblock line toward the top, opposite to the conventional, andreferences the macroblock one to the left and the macroblock one belowthe macroblock to be processed, as peripheral macroblocks.

Further, at the point that encoding of the bottommost macroblock line ofthe odd-numbered regions form the top has ended, the lossless encodingunit 106 starts encoding processing of the region one below(even-numbered from the top).

Also, at this time, the lossless encoding unit 106 references thebottommost macroblock line in the odd-numbered region from the top, andperforms encoding processing for the topmost macroblock line of theregion one below (even numbered from the top).

[Configuration of Lossless Decoding Unit]

FIG. 24 illustrates a configuration example of the lossless decodingunit 202 of the image decoding device 200 in the case of dividing animage into four.

As shown in FIG. 24, the lossless decoding unit 202 in this case has ademultiplexer 551 which divides the encoded data into four slice,instead of the demultiplexer 251, as compared with the case shown inFIG. 10, and further has a slice 3 processing unit 552 and a slice 4processing unit 553.

The slice 3 processing unit 552 is a processing unit for decodingencoded data of slice 3, has the same configuration as with the slice 1processing unit 252, and performs the same processing. With slice 3,encoding has been performed with the same method as slice 1, so theslice 3 processing unit 552 can correctly decode the encoded data ofslice 3 by decoding with the same method as with the slice 1 processingunit 252.

The slice 4 processing unit 553 is a processing unit for decodingencoded data of slice 4, has the same configuration as with the slice 2processing unit 253, and performs the same processing. With slice 4,encoding has been performed with the same method as slice 2, so theslice 4 processing unit 553 can correctly decode the encoded data ofslice 4 by decoding with the same method as with the slice 2 processingunit 253.

First, decoding processing of the encoded data of slice 1 by the slice 1processing unit 252, and decoding processing of the encoded data ofslice 3 by the slice 3 processing unit 552, are started. The decodingprocessing of the topmost macroblock line in slice 2 by the slice 2processing unit 253 is started after waiting for the decoding processingresults of the bottommost macroblock line of slice 1 (referred to asperipheral macroblock).

In the same way, the decoding processing of the topmost macroblock linein slice 4 by the slice 4 processing unit 553 is started after waitingfor the decoding processing results of the bottommost macroblock line ofslice 3 (referred to as peripheral macroblock).

That is to say, the lossless decoding unit 202 of the image decodingdevice 200 divides an image before encoded which corresponds to theencoded data to be decoded, into multiple regions arrayed vertically.The lossless decoding unit 202 then decodes encoded data correspondingto even-numbered regions from the top in the order from the topmacroblock line toward the bottom, as with the conventional, andreferences the macroblock one to the left and the macroblock one abovethe macroblock to be processed, as peripheral macroblocks.

On the other hand, the lossless decoding unit 202 decodes encoded datacorresponding to odd-numbered regions from the top in the order from thebottom macroblock line toward the top, opposite the conventional, andreferences the macroblock one to the left and the macroblock one belowthe macroblock to be processed, as peripheral macroblocks.

Further, at the point that decoding encoded data corresponding to thebottommost macroblock line of an odd-numbered region from the top hasended, the lossless decoding unit 202 starts decoding processing forencoded data of the region one below (even-numbered from the top).

Also, at this time, the lossless decoding unit 202 references thebottommost macroblock line in an odd-numbered region from the top andperforms decoding processing regarding the encoded data corresponding tothe topmost macroblock line of the region one below (even-numbered fromthe top).

Note that an arrangement may be made where the image encoding device 100and image decoding device 200 each have multiple slice processing unitsto begin with, and slice processing units of a number the same as thenumber of entropy slices are operated. That is to say, the number ofentropy slices can be changed in increments of images, increments ofsequences, increments of contents, and so forth. In this case, the imageencoding device 100 notifies the image decoding device 200 of the numberof entropy slices employed for the encoding processing. The imagedecoding device 200 performs decoding processing based on thisinformation, and thus can correctly decode the encoded data which theimage encoding device 100 has generated.

[Macroblocks]

While the size of the macroblocks may be 16×16 or smaller, the size maybe greater than 16×16.

The present technology can be applied to various sizes of macroblockssuch as shown in FIG. 25, for example. For example, the presenttechnology can be applied to not only macroblocks such as normal 16×16pixels, but also to extended macroblocks such as 32×32 pixels (extendedmacroblock).

In FIG. 25, macroblocks made up of 32×32 pixels divided into blocks(partitions) of 32×32 pixels, 32×16 pixels, 16×32 pixels, and 16×16pixels are shown from the left in order on the upper tier. Also, blocksmade up of 16×16 pixels divided into blocks of 16×16 pixels, 16×8pixels, 8×16 pixels, and 8×8 pixels are shown from the left in order onthe middle tier. Also, blocks made up of 8×8 pixels divided into blocksof 8×8 pixels, 8×4 pixels, 4×8 pixels, and 4×4 pixels are shown from theleft in order on the lower tier.

In other words, the macroblocks of 32×32 pixels may be processed withblocks of 32×32 pixels, 32×16 pixels, 16×32 pixels, and 16×16 pixelsshown on the upper tier.

The blocks of 16×16 pixels shown on the right side on the upper tier maybe processed with blocks of 16×16 pixels, 16×8 pixels, 8×16 pixels, and8×8 pixels shown on the middle tier in the same way as with theH.264/AVC format.

The block of 8×8 pixels shown on the right side on the middle tier maybe processed with blocks of 8×8 pixels, 8×4 pixels, 4×8 pixels, and 4×4pixels shown on the lower tier in the same way as with the H.264/AVCformat.

These blocks can be classified into the following three hierarchicallevels. That is, we will refer to the blocks of 32×32 pixels, 32×16pixels and 16×32 pixels, shown on the upper tier in FIG. 25, as a firsthierarchical level. We will refer to the blocks of 16×16 pixels, shownto the right of the upper tier, and the blocks of 16×16 pixels, 16×8pixels, and 8×16 pixels, shown in the middle tier, as a secondhierarchical level. We will refer to the blocks of 8×8 pixels, shown tothe right of the middle tier, and the blocks of 8×8 pixels, 8×4 pixels,4×8 pixels, and 4×4 pixels shown in the lower tier, as a thirdhierarchical level.

By employing such a hierarchical structure, regarding a 16×16 pixelblock or smaller, a greater block can be defined as a superset thereofwhile maintaining compatibility with the H.264/AVC format.

4. Fourth Embodiment [Personal Computer]

The above-described series of processing may be executed by hardware,and may be executed by software. In this case, a configuration may bemade as a personal computer such as shown in FIG. 26, for example.

In FIG. 26, a CPU (Central Processing Unit) 601 of a personal computer600 executes various types of processing following programs stored inROM (Read Only Memory) 602 or programs loaded to RAM (Random AccessMemory) 603 from a storage unit 613. The RAM 603 also stores data and soforth necessary for the CPU 601 to execute various types of processing,as appropriate.

The CPU 601, ROM 602, and RAM 603 are mutually connected by a bus 604.This bus 604 is also connected to an input/output interface 610.

Connected to the input/output interface 610 is an input unit 611 made upof a keyboard, a mouse, and so forth, an output unit 612 made up of adisplay such as a CRT (Cathode Ray Tube) or LCD (Liquid Crystal Display)or the like, a speaker, and so forth, a storage unit 613 made up of ahard disk and so forth, and a communication unit 614 made up of a modemand so forth. The communication unit 614 performs communicationprocessing via networks including the Internet.

Also connected to the input/output interface 610 is a drive 615 asnecessary, to which a removable medium 621 such as a magnetic disk, anoptical disc, a magneto-optical disk, semiconductor memory, or the like,is mounted as appropriate, and computer programs read out therefrom areinstalled in the storage unit 613 as necessary.

In the event of executing the above-described series of processing bysoftware, a program configuring the software is installed from a networkor recording medium.

As shown in FIG. 26, for example, this recording medium is not onlyconfigured of a removable medium 621 made up of a magnetic disk(including flexible disk), optical disc (including CD-ROM (CompactDisc-Read Only Memory), DVD (Digital Versatile Disc), magneto-opticaldisc (including MD (Mini Disc)), or semiconductor memory or the like, inwhich programs are recorded and distributed so as to distribute programsto users separately from the device main unit, but also is configured ofROM 602, a hard disk included in the storage unit 613, and so forth, inwhich programs are recorded, distributed to users in a state of havingbeen built into the device main unit beforehand.

Note that a program which the computer executes may be a program inwhich processing is performed in time sequence following the orderdescribed in the present Specification, or may be a program in whichprocessing is performed in parallel, or at a necessary timing, such aswhen a call-up has been performed.

Also, with the present Specification, steps describing programs recordedin the recording medium includes processing performed in time sequencefollowing the described order as a matter of course, and also processingexecuted in parallel or individually, without necessarily beingprocessed in time sequence.

Also, with the present specification, the term system represents theentirety of devices configured of multiple devices (devices).

Also, a configuration which has been described above as one device (orprocessing unit) may be divided and configured as multiple devices (orprocessing units). Conversely, configurations which have been describedabove as multiple devices (or processing units) may be integrated andconfigured as a single device (or processing unit). Also, configurationsother than those described above may be added to the configuration ofthe devices (or processing units), as a matter of course. Further, partof a configuration of a certain device (or processing unit) may beincluded in a configuration of another device (or another processingunit), as long as the configuration and operations of the overall systemis substantially the same. That is to say, the embodiments of thepresent invention are not restricted to the above-described embodiments,and that various modifications may be made without departing from theessence of the present invention.

For example, the above-described image encoding device 100 and imagedecoding device 200 may be applied to various electronic devices. Thefollowing is a description of examples thereof.

5. Fifth Embodiment [Television Receiver]

FIG. 27 is a block diagram illustrating a primary configuration exampleof a television receiver using the image decoding device 200.

A television receiver 1000 shown in FIG. 27 includes a terrestrial tuner1013, a video decoder 1015, a video signal processing circuit 1018, agraphics generating circuit 1019, a panel driving circuit 1020, and adisplay panel 1021.

The terrestrial tuner 1013 receives the broadcast wave signals of aterrestrial analog broadcast via an antenna, demodulates, obtains videosignals, and supplies these to the video decoder 1015. The video decoder1015 subjects the video signals supplied from the terrestrial tuner 1013to decoding processing, and supplies the obtained digital componentsignals to the video signal processing circuit 1018.

The video signal processing circuit 1018 subjects the video datasupplied from the video decoder 1015 to predetermined processing such asnoise removal or the like, and supplies the obtained video data to thegraphics generating circuit 1019.

The graphics generating circuit 1019 generates the video data of aprogram to be displayed on a display panel 1021, or image data due toprocessing based on an application to be supplied via a network, or thelike, and supplies the generated video data or image data to the paneldriving circuit 1020. Also, the graphics generating circuit 1019 alsoperforms processing such as supplying video data obtained by generatingvideo data (graphics) for the user displaying a screen used forselection of an item or the like, and superimposing this on the videodata of a program, to the panel driving circuit 1020 as appropriate.

The panel driving circuit 1020 drives the display panel 1021 based onthe data supplied from the graphics generating circuit 1019 to displaythe video of a program, or the above-mentioned various screens on thedisplay panel 1021.

The display panel 1021 is made up of an LCD (Liquid Crystal Display) andso forth, and displays the video of a program or the like in accordancewith the control by the panel driving circuit 1020.

Also, the television receiver 1000 also includes an audio A/D(Analog/Digital) conversion circuit 1014, an audio signal processingcircuit 1022, an echo cancellation/audio synthesizing circuit 1023, anaudio amplifier circuit 1024, and a speaker 1025.

The terrestrial tuner 1013 demodulates the received broadcast wavesignal, thereby obtaining not only a video signal but also an audiosignal. The terrestrial tuner 1013 supplies the obtained audio signal tothe audio A/D conversion circuit 1014.

The audio A/D conversion circuit 1014 subjects the audio signal suppliedfrom the terrestrial tuner 1013 to A/D conversion processing, andsupplies the obtained digital audio signal to the audio signalprocessing circuit 1022.

The audio signal processing circuit 1022 subjects the audio datasupplied from the audio A/D conversion circuit 1014 to predeterminedprocessing such as noise removal or the like, and supplies the obtainedaudio data to the echo cancellation/audio synthesizing circuit 1023.

The echo cancellation/audio synthesizing circuit 1023 supplies the audiodata supplied from the audio signal processing circuit 1022 to the audioamplifier circuit 1024.

The audio amplifier circuit 1024 subjects the audio data supplied fromthe echo cancellation/audio synthesizing circuit 1023 to D/A conversionprocessing, subjects to amplifier processing to adjust to predeterminedvolume, and then outputs the audio from the speaker 1025.

Further, the television receiver 1000 also includes a digital tuner1016, and an MPEG decoder 1017.

The digital tuner 1016 receives the broadcast wave signals of a digitalbroadcast (terrestrial digital broadcast, BS (Broadcasting Satellite)/CS(Communications Satellite) digital broadcast) via the antenna,demodulates to obtain MPEG-TS (Moving Picture Experts Group-TransportStream), and supplies this to the MPEG decoder 1017.

The MPEG decoder 1017 descrambles the scrambling given to the MPEG-TSsupplied from the digital tuner 1016, and extracts a stream includingthe data of a program serving as a playing object (viewing object). TheMPEG decoder 1017 decodes an audio packet making up the extractedstream, supplies the obtained audio data to the audio signal processingcircuit 1022, and also decodes a video packet making up the stream, andsupplies the obtained video data to the video signal processing circuit1018. Also, the MPEG decoder 1017 supplies EPG (Electronic ProgramGuide) data extracted from the MPEG-TS to a CPU 1032 via an unshownpath.

The television receiver 1000 uses the above-mentioned image decodingdevice 200 as the MPEG decoder 1017 for decoding video packets in thisway. Note that the MPEG-TS transmitted from the broadcasting station orthe like has been encoded by the image encoding device 100.

In the same way as with the image decoding device 200, the MPEG decoder1017 decodes encoded data supplied from the image encoding device 100for each entropy slice in parallel, and generates decoded image data. Atthis time, in the same way as with the image decoding device 200, theMPEG decoder 1017 divides the image before encoding, corresponding tothe encoded data to be decoded, into vertically arrayed multiple (aneven number) regions. The MPEG decoder 1017 then performs decoding inthe conventional manner in the order from the top macroblock line towardthe bottom for the encoded data corresponding to even-numbered regionsfrom the top, referencing as peripheral macroblocks the macroblock oneto the left and one above the macroblock to be processed.

Conversely, the MPEG decoder 1017 performs decoding opposite theconventional manner, in the order from the bottom macroblock line towardthe top, for the encoded data corresponding to odd-numbered regions fromthe top, referencing as peripheral macroblocks the macroblock one to theleft and one below the macroblock to be processed.

Further, at the point that decoding of encoded data corresponding to thebottommost macroblock line of the odd-numbered regions from the topending, the MPEG decoder 1017 starts decoding processing for the encodeddata of the region one thereunder (even-numbered from the top).

Also, at this time, the MPEG decoder 1017 references the bottommostmacroblock line of the even-numbered regions from the top to performdecoding processing for the encoded data corresponding to the topmostmacroblock line of the region one thereunder (even-numbered from thetop).

Accordingly, the MPEG decoder 1017 can realize high speed image encodingwhile suppressing deterioration in encoding efficiency.

The video data supplied from the MPEG decoder 1017 is, in the same wayas with the case of the video data supplied from the video decoder 1015,subjected to predetermined processing at the video signal processingcircuit 1018, superimposed on the generated video data and so forth atthe graphics generating circuit 1019 as appropriate, supplied to thedisplay panel 1021 via the panel driving circuit 1020, and the imagethereof is displayed thereon.

The audio data supplied from the MPEG decoder 1017 is, in the same wayas with the case of the audio data supplied from the audio A/Dconversion circuit 1014, subjected to predetermined processing at theaudio signal processing circuit 1022, supplied to the audio amplifiercircuit 1024 via the echo cancellation/audio synthesizing circuit 1023,and subjected to D/A conversion processing and amplifier processing. Asa result thereof, the audio adjusted in predetermined volume is outputfrom the speaker 1025.

Also, the television receiver 1000 also includes a microphone 1026, andan A/D conversion circuit 1027.

The A/D conversion circuit 1027 receives the user's audio signalscollected by the microphone 1026 provided to the television receiver1000 serving as for audio conversation, subjects the received audiosignal to A/D conversion processing, and supplies the obtained digitalaudio data to the echo cancellation/audio synthesizing circuit 1023.

In the event that the user (user A)'s audio data of the televisionreceiver 1000 has been supplied from the A/D conversion circuit 1027,the echo cancellation/audio synthesizing circuit 1023 perform echocancellation with the user (user A)'s audio data taken as a object, andoutputs audio data obtained by synthesizing the user A's audio data andother audio data, or the like from the speaker 1025 via the audioamplifier circuit 1024.

Further, the television receiver 1000 also includes an audio codec 1028,an internal bus 1029, SDRAM (Synchronous Dynamic Random Access Memory)1030, flash memory 1031, a CPU 1032, a USB (Universal Serial Bus) I/F1033, and a network I/F 1034.

The A/D conversion circuit 1027 receives the user's audio signalcollected by the microphone 1026 provided to the television receiver1000 serving as for audio conversation, subjects the received audiosignal to A/D conversion processing, and supplies the obtained digitalaudio data to the audio codec 1028.

The audio codec 1028 converts the audio data supplied from the A/Dconversion circuit 1027 into the data of a predetermined format fortransmission via a network, and supplies to the network I/F 1034 via theinternal bus 1029.

The network I/F 1034 is connected to the network via a cable mounted ona network terminal 1035. The network I/F 1034 transmits the audio datasupplied from the audio codec 1028 to another device connected to thenetwork thereof, for example. Also, the network I/F 1034 receives, viathe network terminal 1035, the audio data transmitted from anotherdevice connected thereto via the network, and supplies this to the audiocodec 1028 via the internal bus 1029, for example.

The audio codec 1028 converts the audio data supplied from the networkI/F 1034 into the data of a predetermined format, and supplies this tothe echo cancellation/audio synthesizing circuit 1023.

The echo cancellation/audio synthesizing circuit 1023 performs echocancellation with the audio data supplied from the audio codec 1028taken as a object, and outputs the data of audio obtained bysynthesizing the audio data and other audio data, or the like, from thespeaker 1025 via the audio amplifier circuit 1024.

The SDRAM 1030 stores various types of data necessary for the CPU 1032performing processing.

The flash memory 1031 stores a program to be executed by the CPU 1032.The program stored in the flash memory 1031 is read out by the CPU 1032at predetermined timing such as when activating the television receiver1000, or the like. EPG data obtained via a digital broadcast, dataobtained from a predetermined server via the network, and so forth arealso stored in the flash memory 1031.

For example, MPEG-TS including the content data obtained from apredetermined server via the network by the control of the CPU 1032 isstored in the flash memory 1031. The flash memory 1031 supplies theMPEG-TS thereof to the MPEG decoder 1017 via the internal bus 1029 bythe control of the CPU 1032, for example.

The MPEG decoder 1017 processes the MPEG-TS thereof in the same way aswith the case of the MPEG-TS supplied from the digital tuner 1016. Inthis way, the television receiver 1000 receives the content data made upof video, audio, and so forth via the network, decodes using the MPEGdecoder 1017, whereby video thereof can be displayed, and audio thereofcan be output.

Also, the television receiver 1000 also includes a light reception unit1037 for receiving the infrared signal transmitted from a remotecontroller 1051.

The light reception unit 1037 receives infrared rays from the remotecontroller 1051, and outputs a control code representing the content ofthe user's operation obtained by demodulation, to the CPU 1032.

The CPU 1032 executes the program stored in the flash memory 1031 tocontrol the entire operation of the television receiver 1000 accordingto the control code supplied from the light reception unit 1037, and soforth. The CPU 1032, and the units of the television receiver 1000 areconnected via an unshown path.

The USB I/F 1033 performs transmission/reception of data as to anexternal device of the television receiver 1000 which is connected via aUSB cable mounted on a USB terminal 1036. The network I/F 1034 connectsto the network via a cable mounted on the network terminal 1035, alsoperforms transmission/reception of data other than audio data as tovarious devices connected to the network.

The television receiver 1000 can correctly decode encoded data encodedat highs peed, while suppressing deterioration in encoding efficiency,by using the image decoding device 200 as the MPEG decoder 1017. As aresult, the television receiver 1000 can further improve encodingefficiency of broadcast signals received via an antenna or content dataobtained via a network, and can realize real-time processing with lowercosts.

6. Sixth Embodiment [Cellular Telephone]

FIG. 28 is a block diagram illustrating a primary configuration exampleof a cellular telephone using the image encoding device 100 and imagedecoding device 200.

A cellular telephone 1100 shown in FIG. 28 includes a main control unit1150 configured so as to integrally control the units, a power supplycircuit unit 1151, an operation input control unit 1152, an imageencoder 1153, a camera I/F unit 1154, an LCD control unit 1155, an imagedecoder 1156, a multiplexing/separating unit 1157, a recording/playingunit 1162, a modulation/demodulation circuit unit 1158, and an audiocodec 1159. These are mutually connected via a bus 1160.

Also, the cellular telephone 1100 includes operation keys 1119, a CCD(Charge Coupled Devices) camera 1116, a liquid crystal display 1118, astorage unit 1123, a transmission/reception circuit unit 1163, anantenna 1114, a microphone (MIC) 1121, and a speaker 1117.

Upon a call end and power key being turned on by the user's operation,the power supply circuit unit 1151 activates the cellular telephone 1100in an operational state by supplying power to the units from a batterypack.

The cellular telephone 1100 performs various operations, such astransmission/reception of an audio signal, transmission/reception of ane-mail and image data, image shooting, data recoding, and so forth, invarious modes such as a voice call mode, a data communication mode, andso forth, based on the control of the main control unit 1150 made up ofa CPU, ROM, RAM, and so forth.

For example, in the voice call mode, the cellular telephone 1100converts the audio signal collected by the microphone (mike) 1121 intodigital audio data by the audio codec 1159, subjects this to spectrumspread processing at the modulation/demodulation circuit unit 1158, andsubjects this to digital/analog conversion processing and frequencyconversion processing at the transmission/reception circuit unit 1163.The cellular telephone 1100 transmits the signal for transmissionobtained by the conversion processing thereof to an unshown base stationvia the antenna 1114. The signal for transmission (audio signal)transmitted to the base station is supplied to the cellular telephone ofthe other party via the public telephone network.

Also, for example, in the voice call mode, the cellular telephone 1100amplifies the reception signal received at the antenna 1114, at thetransmission/reception circuit unit 1163, further subjects to frequencyconversion processing and analog/digital conversion processing, subjectsto spectrum inverse spread processing at the modulation/demodulationcircuit unit 1158, and converts into an analog audio signal by the audiocodec 1159. The cellular telephone 1100 outputs the converted andobtained analog audio signal thereof from the speaker 1117.

Further, for example, in the event of transmitting an e-mail in the datacommunication mode, the cellular telephone 1100 accepts the text data ofthe e-mail input by the operation of the operation keys 1119 at theoperation input control unit 1152. The cellular telephone 1100 processesthe text data thereof at the main control unit 1150, and displays on theliquid crystal display 1118 via the LCD control unit 1155 as an image.

Also, the cellular telephone 1100 generates e-mail data at the maincontrol unit 1150 based on the text data accepted by the operation inputcontrol unit 1152, the user's instructions, and so forth. The cellulartelephone 1100 subjects the e-mail data thereof to spectrum spreadprocessing at the modulation/demodulation circuit unit 1158, andsubjects to digital/analog conversion processing and frequencyconversion processing at the transmission/reception circuit unit 1163.The cellular telephone 1100 transmits the signal for transmissionobtained by the conversion processing thereof to an unshown base stationvia the antenna 1114. The signal for transmission (e-mail) transmittedto the base station is supplied to a predetermined destination via thenetwork, mail server, and so forth.

Also, for example, in the event of receiving an e-mail in the datacommunication mode, the cellular telephone 1100 receives the signaltransmitted from the base station via the antenna 1114 with thetransmission/reception circuit unit 1163, amplifies, and furthersubjects to frequency conversion processing and analog/digitalconversion processing. The cellular telephone 1100 subjects thereception signal thereof to spectrum inverse spread processing at themodulation/demodulation circuit unit 1158 to restore the original e-maildata. The cellular telephone 1100 displays the restored e-mail data onthe liquid crystal display 1118 via the LCD control unit 1155.

Note that the cellular telephone 1100 may record (store) the receivede-mail data in the storage unit 1123 via the recording/playing unit1162.

This storage unit 1123 is an optional rewritable recording medium. Thestorage unit 1123 may be, for example, semiconductor memory such as RAM,built-in flash memory, or the like, may be a hard disk, or may be aremovable medium such as a magnetic disk, a magneto-optical disk, anoptical disc, USB memory, a memory card, or the like. It goes withoutsaying that the storage unit 1123 may be other than these.

Further, for example, in the event of transmitting image data in thedata communication mode, the cellular telephone 1100 generates imagedata by imaging at the CCD camera 1116. The CCD camera 1116 includes aCCD serving as an optical device such as a lens, diaphragm, and soforth, and serving as a photoelectric conversion device, which images asubject, converts the intensity of received light into an electricalsignal, and generates the image data of an image of the subject. The CCDcamera 1116 performs encoding of the image data at the image encoder1153 via the camera I/F unit 1154, and converts into encoded image data.

The cellular telephone 1100 employs the above-mentioned image encodingdevice 100 as the image encoder 1153 for performing such processing. Inthe same way as with the image encoding device 100, the image encoder1153 divides the image to be encoded into vertically arrayed multiple(an even number) regions. The image encoder 1153 then performs encodingin the conventional manner in the order from the top macroblock linetoward the bottom corresponding to even-numbered regions from the top,referencing as peripheral macroblocks the macroblock one to the left andone above the macroblock to be processed.

Conversely, the image encoder 1153 performs encoding opposite theconventional manner, in the order from the bottom macroblock line towardthe top, for odd-numbered regions from the top, referencing asperipheral macroblocks the macroblock one to the left and one below themacroblock to be processed.

Further, at the point that encoding has ended for the bottommostmacroblock line of the odd-numbered regions from the top ending, theimage encoder 1153 starts decoding processing for the encoded data ofthe region one thereunder (even-numbered from the top).

Also, at this time, the image encoder 1153 references the bottommostmacroblock line of the even-numbered regions from the top to performencoding processing for the topmost macroblock line of the region onethereunder (even-numbered from the top).

Accordingly, the image encoder 1153 can realize high speed imageencoding while suppressing deterioration in encoding efficiency.

Note that, at this time simultaneously, the cellular telephone 1100converts the audio collected at the microphone (mike) 1121, whileshooting with the CCD camera 1116, from analog to digital at the audiocodec 1159, and further encodes this.

The cellular telephone 1100 multiplexes the encoded image data suppliedfrom the image encoder 1153, and the digital audio data supplied fromthe audio codec 1159 at the multiplexing/separating unit 1157 using apredetermined method. The cellular telephone 1100 subjects themultiplexed data obtained as a result thereof to spectrum spreadprocessing at the modulation/demodulation circuit unit 1158, andsubjects to digital/analog conversion processing and frequencyconversion processing at the transmission/reception circuit unit 1163.The cellular telephone 1100 transmits the signal for transmissionobtained by the conversion processing thereof to an unshown base stationvia the antenna 1114. The signal for transmission (image data)transmitted to the base station is supplied to the other party via thenetwork or the like.

Note that in the event that image data is not transmitted, the cellulartelephone 1100 may also display the image data generated at the CCDcamera 1116 on the liquid crystal display 1118 via the LCD control unit1155 instead of the image encoder 1153.

Also, for example, in the event of receiving the data of a moving imagefile linked to a simple website or the like in the data communicationmode, the cellular telephone 1100 receives the signal transmitted fromthe base station at the transmission/reception circuit unit 1163 via theantenna 1114, amplifies, and further subjects to frequency conversionprocessing and analog/digital conversion processing. The cellulartelephone 1100 subjects the received signal to spectrum inverse spreadprocessing at the modulation/demodulation circuit unit 1158 to restorethe original multiplexed data. The cellular telephone 1100 separates themultiplexed data thereof at the multiplexing/separating unit 1157 intoencoded image data and audio data.

The cellular telephone 1100 decodes the encoded image data at the imagedecoder 1156, thereby generating playing moving image data, and displaysthis on the liquid crystal display 1118 via the LCD control unit 1155.Thus, moving image data included in a moving image file linked to asimple website is displayed on the liquid crystal display 1118, forexample.

The cellular telephone 1100 employs the above-mentioned image decodingdevice 200 as the image decoder 1156 for performing such processing.That is to say, in the same way as with the image decoding device 200,the image decoder 1156 decodes encoded data supplied from the imageencoding device 100 for each entropy slice in parallel, and generatesdecoded image data. At this time, in the same way as with the imagedecoding device 200, the image decoder 1156 divides the image beforeencoding, corresponding to the encoded data to be decoded, intovertically arrayed multiple (an even number) regions. The image decoder1156 then performs decoding in the conventional manner in the order fromthe top macroblock line toward the bottom for the encoded datacorresponding to even-numbered regions from the top, referencing asperipheral macroblocks the macroblock one to the left and one above themacroblock to be processed.

Conversely, the image decoder 1156 performs decoding opposite theconventional manner, in the order from the bottom macroblock line towardthe top, for the encoded data corresponding to odd-numbered regions fromthe top, referencing as peripheral macroblocks the macroblock one to theleft and one below the macroblock to be processed.

Further, at the point that decoding of encoded data corresponding to thebottommost macroblock line of the odd-numbered regions from the topending, the image decoder 1156 starts decoding processing for theencoded data of the region one thereunder (even-numbered from the top).

Also, at this time, the image decoder 1156 references the bottommostmacroblock line of the odd-numbered regions from the top to performdecoding processing for the encoded data corresponding to the topmostmacroblock line of the region one thereunder (even-numbered from thetop).

Accordingly, the image decoder 1156 can realize high speed imageencoding while suppressing deterioration in encoding efficiency.

At this time, simultaneously, the cellular telephone 1100 converts thedigital audio data into an analog audio signal at the audio codec 1159,and outputs this from the speaker 1117. Thus, audio data included in amoving image file linked to a simple website is played, for example.

Note that, in the same way as with the case of e-mail, the cellulartelephone 1100 may record (store) the received data linked to a simplewebsite or the like in the storage unit 1123 via the recording/playingunit 1162.

Also, the cellular telephone 1100 analyzes the imaged two-dimensionalcode obtained by the CCD camera 1116 at the main control unit 1150,whereby information recorded in the two-dimensional code can beobtained.

Further, the cellular telephone 1100 can communicate with an externaldevice at the infrared communication unit 1181 using infrared rays.

The cellular telephone 1100 employs the image encoding device 100 as theimage encoder 1153, whereby deterioration of encoding efficiency due tohigh speed of image encoding can be suppressed at the time of encodingand transmitting image data generated at the CCD camera 1116, forexample, thereby realizing real-time processing at lower costs.

Also, the cellular telephone 1100 employs the image decoding device 200as the image decoder 1156, and thus can correctly decode data of amoving image file linked to at a simple website or the like (encodeddata encoded at high speed while suppressing deterioration in encodingefficiency), for example, thereby realizing real-time processing atlower costs.

Note that description has been made so far wherein the cellulartelephone 1100 employs the CCD camera 1116, but the cellular telephone1100 may employ an image sensor (CMOS image sensor) using CMOS(Complementary Metal Oxide Semiconductor) instead of this CCD camera1116. In this case as well, the cellular telephone 1100 can image asubject and generate the image data of an image of the subject in thesame way as with the case of employing the CCD camera 1116.

Also, description has been made so far regarding the cellular telephone1100, but the image encoding device 100 and the image decoding device200 may be applied to any kind of device in the same way as with thecase of the cellular telephone 1100 as long as it is a device having thesame imaging function and communication function as those of thecellular telephone 1100, for example, such as a PDA (Personal DigitalAssistants), smart phone, UMPC (Ultra Mobile Personal Computer), netbook, notebook-sized personal computer, or the like.

7. Seventh Embodiment [Hard Disk Recorder]

FIG. 29 is a block diagram illustrating a primary configuration exampleof a hard disk recorder which employs the image encoding device 100 andimage decoding device 200 to which the present invention has beenapplied.

A hard disk recorder (HDD recorder) 1200 shown in FIG. 29 is a devicewhich stores, in a built-in hard disk, audio data and video data of abroadcast program included in broadcast wave signals (televisionsignals) received by a tuner and transmitted from a satellite or aterrestrial antenna or the like, and provides the stored data to theuser at timing according to the user's instructions.

The hard disk recorder 1200 can extract audio data and video data frombroadcast wave signals, decode these as appropriate, and store in thebuilt-in hard disk, for example. Also, the hard disk recorder 1200 canalso obtain audio data and video data from another device via thenetwork, decode these as appropriate, and store in the built-in harddisk, for example.

Further, the hard disk recorder 1200 can decode audio data and videodata recorded in the built-in hard disk, supply this to a monitor 1260,display an image thereof on the screen of the monitor 1260, and outputaudio thereof from the speaker of the monitor 1260, for example. Also,the hard disk recorder 1200 can decode audio data and video dataextracted from broadcast signals obtained via a tuner, or audio data andvideo data obtained from another device via a network, supply this tothe monitor 1260, display an image thereof on the screen of the monitor1260, and output audio thereof from the speaker of the monitor 1260, forexample.

Of course, operations other than these may be performed.

As shown in FIG. 29, the hard disk recorder 1200 includes a receptionunit 1221, a demodulation unit 1222, a demultiplexer 1223, an audiodecoder 1224, a video decoder 1225, and a recorder control unit 1226.The hard disk recorder 1200 further includes EPG data memory 1227,program memory 1228, work memory 1229, a display converter 1230, an OSD(On Screen Display) control unit 1231, a display control unit 1232, arecording/playing unit 1233, a D/A converter 1234, and a communicationunit 1235.

Also, the display converter 1230 includes a video encoder 1241. Therecording/playing unit 1233 includes an encoder 1251 and a decoder 1252.

The reception unit 1221 receives the infrared signal from the remotecontroller (not shown), converts into an electrical signal, and outputsto the recorder control unit 1226. The recorder control unit 1226 isconfigured of, for example, a microprocessor and so forth, and executesvarious types of processing in accordance with the program stored in theprogram memory 1228. At this time, the recorder control unit 1226 usesthe work memory 1229 according to need.

The communication unit 1235, which is connected to the network, performscommunication processing with another device via the network. Forexample, the communication unit 1235 is controlled by the recordercontrol unit 1226 to communicate with a tuner (not shown), and toprincipally output a channel selection control signal to the tuner.

The demodulation unit 1222 demodulates the signal supplied from thetuner, and outputs to the demultiplexer 1223. The demultiplexer 1223separates the data supplied from the demodulation unit 1222 into audiodata, video data, and EPG data, and outputs to the audio decoder 1224,video decoder 1225, and recorder control unit 1226, respectively.

The audio decoder 1224 decodes the input audio data, and outputs to therecording/playing unit 1233. The video decoder 1225 decodes the inputvideo data, and outputs to the display converter 1230. The recordercontrol unit 1226 supplies the input EPG data to the EPG data memory1227 for storing.

The display converter 1230 encodes the video data supplied from thevideo decoder 1225 or recorder control unit 1226 into, for example, thevideo data conforming to the NTSC (National Television StandardsCommittee) format using the video encoder 1241, and outputs to therecording/playing unit 1233. Also, the display converter 1230 convertsthe size of the screen of the video data supplied from the video decoder1225 or recorder control unit 1226 into the size corresponding to thesize of the monitor 1260, converts into the video data conforming to theNTSC format using the video encoder 1241, converts into an analogsignal, and outputs to the display control unit 1232.

The display control unit 1232 superimposes, under the control of therecorder control unit 1226, the OSD signal output from the OSD (OnScreen Display) control unit 1231 on the video signal input from thedisplay converter 1230, and outputs to the display of the monitor 1260for display.

Also, the audio data output from the audio decoder 1224 has beenconverted into an analog signal using the D/A converter 1234, andsupplied to the monitor 1260. The monitor 1260 outputs this audio signalfrom a built-in speaker.

The recording/playing unit 1233 includes a hard disk as a recordingmedium in which video data, audio data, and so forth are recorded.

The recording/playing unit 1233 encodes the audio data supplied from theaudio decoder 1224 by the encoder 1251, for example. Also, therecording/playing unit 1233 encodes the video data supplied from thevideo encoder 1241 of the display converter 1230 by the encoder 1251.The recording/playing unit 1233 synthesizes the encoded data of theaudio data thereof, and the encoded data of the video data thereof usingthe multiplexer. The recording/playing unit 1233 amplifies thesynthesized data by channel coding, and writes the data thereof in thehard disk via a recording head.

The recording/playing unit 1233 plays the data recorded in the hard diskvia a playing head, amplifies, and separates into audio data and videodata using the demultiplexer. The recording/playing unit 1233 decodesthe audio data and video data by the decoder 1252. The recording/playingunit 1233 converts the decoded audio data from digital to analog, andoutputs to the speaker of the monitor 1260. Also, the recording/playingunit 1233 converts the decoded video data from digital to analog, andoutputs to the display of the monitor 1260.

The recorder control unit 1226 reads out the latest EPG data from theEPG data memory 1227 based on the user's instructions indicated by theinfrared signal from the remote controller which is received via thereception unit 1221, and supplies this to the OSD control unit 1231. TheOSD control unit 1231 generates image data corresponding to the inputEPG data, and outputs to the display control unit 1232. The displaycontrol unit 1232 outputs the video data input from the OSD control unit1231 to the display of the monitor 1260 for display. Thus, EPG(Electronic Program Guide) is displayed on the display of the monitor1260.

Also, the hard disk recorder 1200 can obtain various types of data suchas video data, audio data, EPG data, and so forth supplied from anotherdevice via the network such as the Internet or the like.

The communication unit 1235 is controlled by the recorder control unit1226 to obtain encoded data such as video data, audio data, EPG data,and so forth transmitted from another device via the network, and tosupply this to the recorder control unit 1226. The recorder control unit1226 supplies the encoded data of the obtained video data and audio datato the recording/playing unit 1233, and stores in the hard disk, forexample. At this time, the recorder control unit 1226 andrecording/playing unit 1233 may perform processing such as re-encodingor the like according to need.

Also, the recorder control unit 1226 decodes the encoded data of theobtained video data and audio data, and supplies the obtained video datato the display converter 1230. The display converter 1230 processes, inthe same way as the video data supplied from the video decoder 1225, thevideo data supplied from the recorder control unit 1226, supplies to themonitor 1260 via the display control unit 1232 for displaying an imagethereof.

Alternatively, an arrangement may be made wherein in accordance withthis image display, the recorder control unit 1226 supplies the decodedaudio data to the monitor 1260 via the D/A converter 1234, and outputsaudio thereof from the speaker.

Further, the recorder control unit 1226 decodes the encoded data of theobtained EPG data, and supplies the decoded EPG data to the EPG datamemory 1227.

The hard disk recorder 1200 thus configured employs the image decodingdevice 200 as the video decoder 1225, decoder 1252, and decoder housedin the recorder control unit 1226. In the same way as with the imagedecoding device 200, the video decoder 1225, decoder 1252, and decoderhoused in the recorder control unit 1226 decode encoded data suppliedfrom the image encoding device 100 for each entropy slice in parallel,and generates decoded image data. At this time, in the same way as withthe image decoding device 200, the video decoder 1225, decoder 1252, anddecoder housed in the recorder control unit 1226 divides the imagebefore encoding, corresponding to the encoded data to be decoded, intovertically arrayed multiple (an even number) regions. The video decoder1225, decoder 1252, and decoder housed in the recorder control unit 1226then performs decoding in the conventional manner in the order from thetop macroblock line toward the bottom for the encoded data correspondingto even-numbered regions from the top, referencing as peripheralmacroblocks the macroblock one to the left and one above the macroblockto be processed.

Conversely, the video decoder 1225, decoder 1252, and decoder housed inthe recorder control unit 1226 performs decoding opposite theconventional manner, in the order from the bottom macroblock line towardthe top, for the encoded data corresponding to odd-numbered regions fromthe top, referencing as peripheral macroblocks the macroblock one to theleft and one below the macroblock to be processed.

Further, at the point that decoding of encoded data corresponding to thebottommost macroblock line of the odd-numbered regions from the topending, the video decoder 1225, decoder 1252, and decoder housed in therecorder control unit 1226 starts decoding processing for the encodeddata of the region one thereunder (even-numbered from the top).

Also, at this time, the video decoder 1225, decoder 1252, and decoderhoused in the recorder control unit 1226 references the bottommostmacroblock line of the odd-numbered regions from the top to performdecoding processing for the encoded data corresponding to the topmostmacroblock line of the region one thereunder (even-numbered from thetop).

Accordingly, the video decoder 1225, decoder 1252, and decoder housed inthe recorder control unit 1226 can realize high speed image encodingwhile suppressing deterioration in encoding efficiency.

Accordingly, the hard disk recorder 1200 can correctly decode video datawhich the tuner or communication unit 1235 receives (encoded dataencoded at high speed while suppressing deterioration in encodingefficiency), and video data which the recording/playing unit 1233 plays(encoded data encoded at high speed while suppressing deterioration inencoding efficiency), for example, thereby realizing real-timeprocessing with lower costs.

Also, the hard disk recorder 1200 employs the image encoding device 100as the encoder 1251. In the same way as with the image encoding device100, the image encoder 1251 divides the image to be encoded intovertically arrayed multiple (an even number) regions. The image encoder1251 then performs encoding in the conventional manner in the order fromthe top macroblock line toward the bottom for the encoded datacorresponding to even-numbered regions from the top, referencing asperipheral macroblocks the macroblock one to the left and one above themacroblock to be processed.

Conversely, the image encoder 1251 performs encoding opposite theconventional manner, in the order from the bottom macroblock line towardthe top, for odd-numbered regions from the top, referencing asperipheral macroblocks the macroblock one to the left and one below themacroblock to be processed.

Further, at the point that encoding has ended for the bottommostmacroblock line of the odd-numbered regions from the top ending, theimage encoder 1251 starts decoding processing for the encoded data ofthe region one thereunder (even-numbered from the top).

Also, at this time, the image encoder 1251 references the bottommostmacroblock line of the odd-numbered regions from the top to performencoding processing for the topmost macroblock line of the region onethereunder (even-numbered from the top).

Accordingly, the image encoder 1251 can realize high speed imageencoding while suppressing deterioration in encoding efficiency.

Accordingly, the hard disk recorder 1200 can suppress deterioration inencoding efficiency due to high speed image encoding at the time ofgenerating encoded data to be recorded in the hard disk. As a result thecellular phone 1100 can realize real-time processing with lower costs.

Note that description has been made so far regarding the hard diskrecorder 1200 for recording video data and audio data in the hard disk,but it goes without saying that any kind of recording medium may beemployed. For example, even with a recorder to which a recording mediumother than a hard disk, such as flash memory, optical disc, video tape,or the like, is applied, the image encoding device 100 and imagedecoding device 200 can be applied thereto in the same way as with thecase of the above hard disk recorder 1200.

8. Eighth Embodiment [Camera]

FIG. 30 is a block diagram illustrating a primary configuration exampleof a camera employing the image encoding device 100 and image decodingdevice 200 to which the present invention has been applied.

A camera 1300 shown in FIG. 30 images a subject, displays an image ofthe subject on an LCD 1316, and records this in a recording medium 1333as image data.

A lens block 1311 inputs light (i.e., picture of a subject) to aCCD/CMOS 1312. The CCD/CMOS 1312 is an image sensor employing a CCD orCMOS, which converts the intensity of received light into an electricalsignal, and supplies to a camera signal processing unit 1313.

The camera signal processing unit 1313 converts the electrical signalsupplied from the CCD/CMOS 1312 into color difference signals of Y, Cr,and Cb, and supplies to an image signal processing unit 1314. The imagesignal processing unit 1314 subjects, under the control of a controller1321, the image signal supplied from the camera signal processing unit1313 to predetermined image processing, or encodes the image signalthereof by an encoder 1341. The image signal processing unit 1314supplies encoded data generated by encoding an image signal, to adecoder 1315. Further, the image signal processing unit 1314 obtainsdata for display generated at an on-screen display (OSD) 1320, andsupplies this to the decoder 1315.

With the above-mentioned processing, the camera signal processing unit1313 appropriately takes advantage of DRAM (Dynamic Random AccessMemory) 1318 connected via a bus 1317 to hold image data, encoded dataencoded from the image data thereof, and so forth in the DRAM 1318thereof according to need.

The decoder 1315 decodes the encoded data supplied from the image signalprocessing unit 1314, and supplies obtained image data (decoded imagedata) to the LCD 1316. Also, the decoder 1315 supplies the data fordisplay supplied from the image signal processing unit 1314 to the LCD1316. The LCD 1316 synthesizes the image of the decoded image data, andthe image of the data for display, supplied from the decoder 1315 asappropriate, and displays a synthesizing image thereof.

The on-screen display 1320 outputs, under the control of the controller1321, data for display such as a menu screen or icon or the like made upof a symbol, characters, or a figure to the image signal processing unit1314 via the bus 1317.

Based on a signal indicating the content commanded by the user using anoperating unit 1322, the controller 1321 executes various types ofprocessing, and also controls the image signal processing unit 1314,DRAM 1318, external interface 1319, on-screen display 1320, media drive1323, and so forth via the bus 1317. Programs, data, and so forthnecessary for the controller 1321 executing various types of processingare stored in FLASH ROM 1324.

For example, the controller 1321 can encode image data stored in theDRAM 1318, or decode encoded data stored in the DRAM 1318 instead of theimage signal processing unit 1314 and decoder 1315. At this time, thecontroller 1321 may perform encoding/decoding processing using the sameformat as the encoding and decoding format of the image signalprocessing unit 1314 and decoder 1315, or may perform encoding/decodingprocessing using a format that neither the image signal processing unit1314 nor the decoder 1315 can handle.

Also, for example, in the event that start of image printing has beeninstructed from the operating unit 1322, the controller 1321 reads outimage data from the DRAM 1318, and supplies this to a printer 1334connected to the external interface 1319 via the bus 1317 for printing.

Further, for example, in the event that image recording has beeninstructed from the operating unit 1322, the controller 1321 reads outencoded data from the DRAM 1318, and supplies this to a recording medium1333 mounted on the media drive 1323 via the bus 1317 for storing.

The recording medium 1333 is an optional readable/writable removablemedium, for example, such as a magnetic disk, a magneto-optical disk, anoptical disc, semiconductor memory, or the like. It goes without sayingthat the recording medium 1333 is also optional regarding the type of aremovable medium, and accordingly may be a tape device, or may be adisc, or may be a memory card. It goes without saying that the recodingmedium 1333 may be a non-contact IC card or the like.

Alternatively, the media drive 1323 and the recording medium 1333 may beconfigured so as to be integrated into a non-transportable recordingmedium, for example, such as a built-in hard disk drive, SSD (SolidState Drive), or the like.

The external interface 1319 is configured of, for example, a USBinput/output terminal and so forth, and is connected to the printer 1334in the event of performing printing of an image. Also, a drive 1331 isconnected to the external interface 1319 according to need, on which theremovable medium 1332 such as a magnetic disk, optical disc, ormagneto-optical disk is mounted as appropriate, and a computer programread out therefrom is installed in the FLASH ROM 1324 according to need.

Further, the external interface 1319 includes a network interface to beconnected to a predetermined network such as a LAN, the Internet, or thelike. For example, in accordance with the instructions from theoperating unit 1322, the controller 1321 can read out encoded data fromthe DRAM 1318, and supply this from the external interface 1319 toanother device connected via the network. Also, the controller 1321 canobtain, via the external interface 1319, encoded data or image datasupplied from another device via the network, and hold this in the DRAM1318, or supply this to the image signal processing unit 1314.

The camera 1300 thus configured employs the image decoding device 200 asthe decoder 1315. That is to say, in the same way as with the imagedecoding device 200, the decoder 1315 decodes encoded data supplied fromthe image encoding device 100 for each entropy slice in parallel, andgenerates decoded image data. At this time, in the same way as with theimage decoding device 200, the decoder 1315 divides the image beforeencoding, corresponding to the encoded data to be decoded, intovertically arrayed multiple (an even number) regions. The decoder 1315then performs decoding in the conventional manner in the order from thetop macroblock line toward the bottom for the encoded data correspondingto even-numbered regions from the top, referencing as peripheralmacroblocks the macroblock one to the left and one above the macroblockto be processed.

Conversely, the decoder 1315 performs decoding opposite the conventionalmanner, in the order from the bottom macroblock line toward the top, forthe encoded data corresponding to odd-numbered regions from the top,referencing as peripheral macroblocks the macroblock one to the left andone below the macroblock to be processed.

Further, at the point that decoding of encoded data corresponding to thebottommost macroblock line of the odd-numbered regions from the topending, the decoder 1315 starts decoding processing for the encoded dataof the region one thereunder (even-numbered from the top).

Also, at this time, the decoder 1315 references the bottommostmacroblock line of the odd-numbered regions from the top to performdecoding processing for the encoded data corresponding to the topmostmacroblock line of the region one thereunder (even-numbered from thetop).

Accordingly, the decoder 1315 can realize high speed image encodingwhile suppressing deterioration in encoding efficiency.

Accordingly, the camera 1300 can further improve encoding efficiency ofimage data generated at the CCD/CMOS 1312, encoded data of video dataread out from the DRAM 1318 or recording medium 1333, and encoded dataof video data obtained via a network for example, thereby realizingreal-time processing with lower costs.

Also, the camera 1300 employs the image encoding device 100 as theencoder 1341. In the same way as with the image encoding device 100, theimage encoder 1341 divides the image to be encoded into verticallyarrayed multiple (an even number) regions. The image encoder 1341 thenperforms encoding in the conventional manner in the order from the topmacroblock line toward the bottom for the encoded data corresponding toeven-numbered regions from the top, referencing as peripheralmacroblocks the macroblock one to the left and one above the macroblockto be processed.

Conversely, the image encoder 1341 performs encoding opposite theconventional manner, in the order from the bottom macroblock line towardthe top, for odd-numbered regions from the top, referencing asperipheral macroblocks the macroblock one to the left and one below themacroblock to be processed.

Further, at the point that encoding has ended for the bottommostmacroblock line of the odd-numbered regions from the top ending, theimage encoder 1341 starts decoding processing for the encoded data ofthe region one thereunder (even-numbered from the top).

Also, at this time, the image encoder 1341 references the bottommostmacroblock line of the odd-numbered regions from the top to performencoding processing for the topmost macroblock line of the region onethereunder (even-numbered from the top).

Accordingly, the image encoder 1341 can realize high speed imageencoding while suppressing deterioration in encoding efficiency.

Accordingly, by using the image encoding device 100 as the encoder 1341,the camera 1300 can suppress deterioration in encoding efficiency due toincreased speed of image encoded of encoded data recorded in the DRAM1318 or recording medium 1333 and encoded data provided to other devicesfor example. As a result thereof, the camera 1300 can realize real-timeprocessing with lower costs.

Note that the decoding method of the image decoding device 200 may beapplied to the decoding processing which the controller 1321 performs.In the same way, the encoding method of the image encoding device 100may be applied to the encoding processing which the controller 1321performs.

Also, the image data which the camera 1300 takes may be moving images ormay be still images.

As a matter of course, the image encoding device 100 and image decodingdevice 200 may be applied to devices or systems other than theabove-described devices.

Note that the present technology may have the following configurations.

(1) An image processing device which encodes image data, the imageprocessing device including:

an encoding unit which, with a small region line in which are arrayed inthe horizontal direction small regions obtained by dividing largeregions arrayed in the vertical direction of an image as an object,performs encoding of image data of the large region, advancing theencoding order of small regions in the same order as with the scanorder, and advancing the encoding order of small region lines in anorder different from the scan order; and

a control unit which controls the encoding unit so as to encode inaccordance with a state of a left small region situated at a leftperiphery of a small region to be processed, and a state of a lowersmall region situated at a lower periphery of the small region to beprocessed.

(2) The image processing device according to (1), wherein the encodingunit, with image data of a large region situated odd-numbered from thetop of the image as an object, advances the encoding order of smallregions in the same order as with the scan order, and advances theencoding order of small region lines in an order different from the scanorder.

(3) The image processing device according to (2), wherein the controlunit, with image data of a large region situated odd-numbered from thetop of the image as an object, controls the encoding unit so as toencode image data of a small region to be processed without referencingthe state of a small region situated one below the small region to beprocessed, for a small region line situated at the bottommost positionof a large region to be processed.

(4) The image processing device according to (3), wherein, in the eventthat image data of a small region line situated at the bottommostposition of a large region to be processed has been stored in a storageunit, the control unit, with image data of a large region situatedodd-numbered from the top of the image as an object, controls theencoding unit so as to read out and encode the image data from thestorage unit in the order of the encoding.

(5) The image processing device according to (4), wherein the encodingunit, with image data of a large region situated even-numbered from thetop of the image as an object, advances the encoding order of smallregions in the same order as with the scan order for the small regionlines, and advances the encoding order of the small region lines in thesame order as with the scan order.

(6) The image processing device according to (5), wherein the controlunit, with image data of a large region situated even-numbered from thetop of the image as an object, for a small region line situated at thetopmost position of a large region to be processed, controls theencoding unit so as to encode image data of a small region to beprocessed while referencing the state of a small region of the smallregion line situated at the bottommost position in a large regionsituated one above the large region.

(7) The image processing device according to (6), wherein, in the eventthat image data of a small region line situated at the topmost positionof a large region to be processed has been stored in a storage unit, thecontrol unit, with image data of a large region situated even-numberedfrom the top of the image as an object, controls the encoding unit so asto read out and encode the image data from the storage unit in the orderof the encoding.

(8) The image processing device according to any one of (1) through (7),wherein the encoding unit advances the encoding order of the smallregion lines in the opposite order from the scan order.

(9) The image processing device according to (8), wherein the scan orderis a raster scan order.

(10) An image processing method of an image processing device whichencodes image data, wherein

an encoding unit, with a small region line in which are arrayed in thehorizontal direction small regions obtained by dividing large regionsarrayed in the vertical direction of an image as an object, performsencoding of image data of the large region, advancing the encoding orderof small regions in the same order as with the scan order, and advancingthe encoding order of small region lines in an order different from thescan order; and

a control unit controls the encoding unit so as to encode in accordancewith a state of a left small region situated at a left periphery of asmall region to be processed, and a state of a lower small regionsituated at a lower periphery of the small region to be processed.

(11) An image processing device which decodes encoded data of image datawhich has been encoded, the image processing device including:

a decoding unit which, with a small region line in which are arrayed inthe horizontal direction small regions obtained by dividing largeregions arrayed in the vertical direction of an image to which theencoded data corresponds as an object, performs decoding of encoded dataof the large region, advancing the decoding order of small regions inthe same order as with the scan order, and advancing the decoding orderof small region lines in an order different from the scan order; and

a control unit which controls the decoding unit so as to decode inaccordance with a state of a left small region situated at a leftperiphery of a small region to be processed, and a state of a lowersmall region situated at a lower periphery of the small region to beprocessed.

(12) The image processing device according to (11), wherein the decodingunit, with image data of a large region situated odd-numbered from thetop of the image as an object, advances the decoding order of smallregions in the same order as with the scan order, and advances thedecoding order of small region lines in an order different from the scanorder.

(13) The image processing device according to (12), wherein the controlunit, with image data of a large region situated odd-numbered from thetop of the image as an object, controls the decoding unit so as todecode image data of a small region to be processed without referencingthe state of a small region situated one below the small region to beprocessed, for a small region line situated at the bottommost positionof a large region to be processed.

(14) The image processing device according to (13), wherein, in theevent that image data of a small region line situated at the bottommostposition of a large region to be processed has been stored in a storageunit, the control unit, with image data of a large region situatedodd-numbered from the top of the image as an object, controls thedecoding unit so as to read out and decode the image data from thestorage unit in the order of the decoding.

(15) The image processing device according to any one of (11) through(14), wherein the decoding unit, with encoded data of a large regionsituated even-numbered from the top of the image as an object, advancesthe decoding order of small regions in the same order as with the scanorder for the small region lines, and advances the decoding order of thesmall region lines in the same order as with the scan order.

(16) The image processing device according to any one of (11) through(15), wherein the control unit, with encoded data of a large regionsituated even-numbered from the top of the image as an object, for asmall region line situated at the topmost position of a large region tobe processed, controls the decoding unit so as to decode encoded data ofa small region to be processed while referencing the state of a smallregion of the small region line situated at the bottommost position in alarge region situated one above the large region.

(17) The image processing device according to (16), wherein, in theevent that image data of a small region line situated at the topmostposition of a large region to be processed has been stored in a storageunit, the control unit, with image data of a large region situatedeven-numbered from the top of the image as an object, controls thedecoding unit so as to read out and decode the image data from thestorage unit in the order of the decoding.

(18) The image processing device according to any one of (11) through(17), wherein the decoding unit advances the decoding order of the smallregion lines in the opposite order from the scan order.

(19) The image processing device according to (18), wherein the scanorder is a raster scan order.

(20) An image processing method of an image processing device whichdecodes encoded data of image data which has been encoded, wherein

a decoding unit, with a small region line in which are arrayed in thehorizontal direction small regions obtained by dividing large regionsarrayed in the vertical direction of an image to which the encoded datacorresponds as an object, performs decoding of encoded data of the largeregion, advancing the decoding order of small regions in the same orderas with the scan order, and advancing the decoding order of small regionlines in an order different from the scan order; and

a control unit controls the decoding unit so as to decode in accordancewith a state of a left small region situated at a left periphery of asmall region to be processed, and a state of a lower small regionsituated at a lower periphery of the small region to be processed.

REFERENCE SIGNS LIST

-   -   100 image encoding device    -   106 lossless encoding unit    -   151 control unit    -   152 storage unit    -   153 slice 1 processing unit    -   154 slice 2 processing unit    -   200 image decoding device    -   202 lossless encoding unit    -   251 demultiplexer    -   252 slice 1 processing unit    -   253 slice 2 processing unit    -   301 context processing unit    -   501 control unit    -   503 slice 3 processing unit    -   504 slice 4 processing unit    -   551 demultiplexer    -   552 slice 3 processing unit    -   554 slice 4 processing unit

1. An encoding method for encoding image data, comprising: performing arithmetic encoding of block lines of a first slice of the image data, and performing arithmetic encoding of block lines of a second slice of the image data; and performing arithmetic encoding, by circuitry of an image processing device, on a top block of a current block line in the second slice, by using a context updated in case of encoding a previous block of a previous block line, the previous block line being located above adjacent to the current block line, the previous block being different from a top block of the previous block line which is located above adjacent to the top block of the current block line, and the previous block being other than a last block in encoding order in the previous block line in the first slice.
 2. The encoding method according to claim 1, further comprising: performing arithmetic encoding, by the circuitry, so as to use the context used in the arithmetic encoding for the previous block of the previous block line as an initial value.
 3. The encoding method according to claim 2, further comprising: performing arithmetic encoding, by the circuitry, so as to use a stage of a probability table used in the arithmetic encoding for the previous block of the previous block line as the initial value.
 4. The encoding method according to claim 3, wherein the previous block line is a neighboring block line on top of the current block line.
 5. The encoding method according to claim 4, wherein the second slice is a current slice and the first slice is a previous slice.
 6. The encoding method according to claim 4, further comprising: transforming the image data to generate the transformed data; quantizing the transformed date to generate quantized date; and performing arithmetic encoding, by the circuitry, on the top block of the current block line of the quantized date to generate a bit stream.
 7. An image processing device for encoding image data, comprising: circuitry configured to: perform arithmetic encoding of block lines of a first slice of the image data, and performing arithmetic encoding of block lines of a second slice of the image data; and perform arithmetic encoding on a top block of a current block line in the second by using a context updated in case of encoding a previous block of a previous block line, the previous block line being located above adjacent to the current block line, the previous block being different from a top block of the previous block line which is located above adjacent to the top block of the current block line, and the previous block being other than a last block in encoding order in the previous block line in the first slice.
 8. The image processing device according to claim 7, wherein the circuitry is configured to perform arithmetic encoding so as to use the context used in the arithmetic encoding process for the previous block of the previous block line as an initial value.
 9. The image processing device according to claim 8, wherein the circuitry is configured to perform arithmetic encoding so as to use a stage of a probability table used in the arithmetic encoding process for the previous block of the previous block line as the initial value.
 10. The image processing device according to claim 9, wherein the previous block line is a neighboring block line on top of the current block line.
 11. The image processing device according to claim 10, wherein the second slice is a current slice and the first slice is a previous slice.
 12. The image processing device according to claim 10, wherein the circuitry is configured to transform the image data to generate the transformed data; quantize the transformed date to generate quantized date; and perform arithmetic encoding, by the circuitry, on the top block of the current block line of the quantized date to generate a bit stream.
 13. The encoding method according to claim 1, wherein the first slice and the second slice are adjacent at a slice boundary as an object.
 14. The image processing device according to claim 7, wherein the first slice and the second slice are adjacent at a slice boundary as an object.
 15. The encoding method according to claim 1, further comprising: initializing a context for encoding the top block of the current block line in the second slice by the context updated in case of encoding the previous block of the previous block line.
 16. The encoding method according to claim 13, wherein the block lines of the first slice of the bit stream are encoded by performing arithmetic encoding of the block lines of the first slice in an order from bottom to top, and the block lines of the second slice of the bit stream are encoded by performing arithmetic encoding of the block lines of the second slice in an order from top to bottom.
 17. The image processing device according to claim 7, wherein the circuitry is configured to initialize a context for encoding the top block of the current block line in the second slice by the context updated in case of encoding the previous block of the previous block line.
 18. The image processing device according to claim 14, wherein the block lines of the first slice of the bit stream are encoded by performing arithmetic encoding of the block lines of the first slice in an order from bottom to top, and the block lines of the second slice of the bit stream are encoded by performing arithmetic encoding of the block lines of the second slice in an order from top to bottom. 